-- Logs begin at Mon 2022-09-19 10:31:18 UTC, end at Mon 2022-09-19 12:16:42 UTC. -- Sep 19 11:03:58.282920 dv-cinder-822822-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 19 11:03:58.295775 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 19 11:03:58.296350 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 19 11:03:58 2022] *** Sep 19 11:03:58.296350 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 19 11:03:58.296350 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 19 11:03:58.296350 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: nodename: dv-cinder-822822-7-iscsi Sep 19 11:03:58.296350 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: machine: x86_64 Sep 19 11:03:58.296350 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: clock source: unix Sep 19 11:03:58.296350 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: pcre jit disabled Sep 19 11:03:58.296350 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: detected number of CPU cores: 4 Sep 19 11:03:58.296350 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: current working directory: / Sep 19 11:03:58.296350 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: detected binary path: /usr/bin/uwsgi-core Sep 19 11:03:58.296350 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: your processes number limit is 47680 Sep 19 11:03:58.296962 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: your memory page size is 4096 bytes Sep 19 11:03:58.296962 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: detected max file descriptor number: 2048 Sep 19 11:03:58.296962 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: lock engine: pthread robust mutexes Sep 19 11:03:58.296962 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: thunder lock: enabled Sep 19 11:03:58.296962 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 19 11:03:58.296962 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 19 11:03:58.335196 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: Python main interpreter initialized at 0x5560484789c0 Sep 19 11:03:58.335196 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: python threads support enabled Sep 19 11:03:58.335196 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: your server socket listen backlog is limited to 100 connections Sep 19 11:03:58.335196 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: your mercy for graceful operations on workers is 90 seconds Sep 19 11:03:58.335583 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: mapped 403077 bytes (393 KB) for 2 cores Sep 19 11:03:58.335660 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: *** Operational MODE: preforking *** Sep 19 11:03:58.335914 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: *** uWSGI is running in multiple interpreter mode *** Sep 19 11:03:58.335914 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: spawned uWSGI master process (pid: 81724) Sep 19 11:03:58.336134 dv-cinder-822822-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 19 11:03:58.336914 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: spawned uWSGI worker 1 (pid: 81737, cores: 1) Sep 19 11:03:58.337012 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: spawned uWSGI worker 2 (pid: 81738, cores: 1) Sep 19 11:03:58.339210 dv-cinder-822822-7-iscsi devstack@keystone.service[81724]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 19 11:04:02.813528 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 11:04:02.813528 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: from cryptography.utils import int_from_bytes Sep 19 11:04:02.814369 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 11:04:02.814369 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: from cryptography.utils import int_from_bytes Sep 19 11:04:02.816003 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 11:04:02.816003 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: from cryptography.utils import int_from_bytes Sep 19 11:04:02.816467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 11:04:02.816467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: from cryptography.utils import int_from_bytes Sep 19 11:04:02.870472 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 19 11:04:02.870650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 19 11:04:02.870836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] command line args: [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 19 11:04:02.870941 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 19 11:04:02.871026 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 19 11:04:02.871098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 19 11:04:02.871272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] command line args: [] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 19 11:04:02.871372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ================================================================================ {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 19 11:04:02.871554 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 19 11:04:02.871638 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] admin_token = **** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.871734 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ================================================================================ {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 19 11:04:02.871869 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] config_dir = [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.871989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] admin_token = **** {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.872076 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.872215 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] config_source = [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.872294 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] config_dir = [] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.872393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] debug = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.872477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.872584 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] config_source = [] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.872662 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.872845 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] debug = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.872922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.872988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.873147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.873231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.873351 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.873438 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.873598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.873689 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.873827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] list_limit = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.873913 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.874016 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] log_config_append = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.874096 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] list_limit = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.874192 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.874263 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] log_config_append = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.874333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] log_dir = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.874397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.874514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] log_file = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.874588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] log_dir = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.874725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] log_file = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.874804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.874978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.875101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.879692 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.879815 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.879924 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.880006 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.880006 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.880159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.880233 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.880308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.880380 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.880459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.880793 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.880870 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.880946 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.881021 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.881099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.881170 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.881238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.881306 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.881374 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.881448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.881533 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.881617 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.881698 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.881773 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.881848 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.881925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.881925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.882121 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.78/identity {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.882121 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] publish_errors = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.882296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.78/identity {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.882382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.882456 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] publish_errors = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.882525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.882598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.882736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.882811 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.882885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.882960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.883047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.883128 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.883217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] standard_threads = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.883299 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.883376 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.883447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] standard_threads = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.883447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.883579 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.883579 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.883711 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.883786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] use_journal = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.883857 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.883969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] use_json = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.884077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] use_journal = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.884156 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] use_stderr = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.884228 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] use_json = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.884298 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] use_syslog = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.884367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] use_stderr = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.884436 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.884514 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] use_syslog = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.884514 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:04:02.884642 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.884717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.884793 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.885002 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.885081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.885149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.885251 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.885332 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.885458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.885458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.885602 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.885698 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.885775 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.885844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.885912 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.885991 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.886062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.886062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.886191 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.886259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.886327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.886401 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.886476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.886551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.886620 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.886688 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.886772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.886879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.886982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.887050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.887120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.887215 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.887289 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.887381 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.887477 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.887555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.887625 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.887693 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.887761 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.887836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.887907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.887974 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.888042 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.888115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.888185 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.888261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.888331 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.888399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.888472 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.888542 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.888615 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.888766 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.888845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.888921 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.888996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.889066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.889134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.889212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.889288 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.889358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.889426 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.889499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.889568 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.889643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.889719 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.889789 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.889857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.889930 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.890010 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.890079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.890148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.890269 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.890349 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.890510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.890595 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.890742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.890818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.890975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.891090 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.891268 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.891394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.891523 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.891632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.891729 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.891824 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.891952 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.892062 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.892188 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.892263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.892373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.892450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.892586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.892675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.892818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.892891 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.893046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.893137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.893275 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.893361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.893507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.893584 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.893742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.893820 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.893970 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.894056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.894206 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.894289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.894439 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.894520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.894674 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.894750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.894911 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.895029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.895143 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.895276 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.895394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.895468 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.895624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.895705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.895859 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.895938 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.896095 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.896178 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.896326 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.896408 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.896559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.896645 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.896808 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.896886 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.897040 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.897124 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.897273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.897383 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.897513 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.897620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.897750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.897844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.898000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.898080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.898250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.898501 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.898584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.898747 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.898827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.898986 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.899069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.899248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.899374 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.899552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.899627 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.899720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.899805 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.899956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.900034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.900188 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.900266 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.900428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.900512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.900663 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.900748 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.900908 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.900998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.901143 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.901219 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.901376 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.901453 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.901609 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.901762 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.901841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.901911 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.902067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.902147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.902322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.902414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.903090 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.903225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.903362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.903440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.903603 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.903692 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.903839 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.903922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.904081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.904221 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.904317 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.904405 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.904553 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.904633 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.904790 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.904868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.905019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.905104 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.905247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.905323 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.905475 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.905551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.905709 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.905793 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.905936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.906013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.906174 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.906252 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.906407 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.906499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.906652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.906730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.906888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.906965 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.907141 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.907295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.907419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.907506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.907660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.907764 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.907888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] auth.external = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.907966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] auth.external = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.908117 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.908206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.908409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.908489 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.908640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.908717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.908870 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] auth.password = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.908954 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] auth.password = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.909097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] auth.token = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.909174 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] auth.token = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.909335 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.909413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.909570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.909648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.909818 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.909896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.910051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.910163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.910295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.910372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.910545 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.910632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.910820 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.910913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.910999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] credential.caching = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.911079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] credential.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.911276 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.911361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.911523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.911613 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.911770 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.911856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.912012 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.912091 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.912254 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.912332 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.912488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.912574 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.912732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.912809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.912983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.913061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.913218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.913297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.913470 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.913547 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.913751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.913828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.913996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.914078 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.914256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.914333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.914504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.914618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.914755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.914862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.914987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] federation.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.915066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] federation.caching = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.915247 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.915349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.915493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.915593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.915737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.915815 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.915962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.916039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.916208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.916294 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.916442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.916520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.916693 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.916771 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.916926 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.917005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.917181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.917259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.917418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.917496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.917661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.917740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.917894 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.917972 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] identity.caching = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.918218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.918833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.918945 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.919026 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.919215 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.919299 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.919432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.919514 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.919676 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.919756 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.919906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.919985 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.920142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.920228 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.920390 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.920468 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.920660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.920747 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.920902 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.921022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.921137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.921217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.921317 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.921397 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.921566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.921654 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.921817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.921895 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.922065 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.922150 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.922322 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.922409 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.922571 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.922650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.922838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.922916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.923076 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.923162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.923341 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.923466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.923566 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.923645 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.923804 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.925510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.925510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.925794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.925875 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.925946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.926016 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.926099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.926230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.926314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.926437 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.926528 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.926607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.926737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.926863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.926938 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.926938 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.927066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.927136 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.927279 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.927355 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.927425 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.927494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.927570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.927648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.927729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.927801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.927871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.927940 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.928008 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.928084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.928163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.928234 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.928304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.928373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.928442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.928511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.928588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.928660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.928735 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.928807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.928877 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.928946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.929024 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.929096 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.929171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.929242 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.929317 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.929429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.929517 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.929682 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.929761 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.929923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.930002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.930157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.930243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.930449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.930534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.930607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.930677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.930837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.930927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.931094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.931193 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.931347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.931437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.931593 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.931680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.931838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.931917 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.932085 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.932171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.932317 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.932444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.932547 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.932625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.932784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.932865 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.933021 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.933102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.933260 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.933339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.933486 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.933564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.933752 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.933831 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.933999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.934078 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.934245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.934323 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.934495 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.934575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.934743 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.934821 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.934975 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.935053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.935248 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.935331 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.935483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.935561 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.935718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.935795 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.935953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.936033 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.936188 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.936265 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.936429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.936506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.936673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.936753 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.936909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.936987 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.937165 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.937252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.937389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.937464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.937630 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.937709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.937869 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.937948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.938102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.938180 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.938340 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.938428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.938582 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.938660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.938821 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.938901 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.939050 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.939129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.939308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.939388 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.939551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] resource.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.939631 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] resource.caching = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.939798 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.939876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.940046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.940125 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.940283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.940533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.940610 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.940786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.940863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.941022 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.941114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.941258 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.941336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.941495 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.941573 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.941728 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.941814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.941964 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] role.caching = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.942041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.942196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] role.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.942272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] role.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.942435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.942521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] role.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.942662 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.942740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.942908 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.942986 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.943152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.943288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.943414 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.943494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.943660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.943737 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.943903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.944001 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.944142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.944222 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.944386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.944464 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.944630 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.944709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.944859 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.944936 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.945101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.945179 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.945343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.945423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.945587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.945666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.945829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.946007 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.946241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.946321 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.946477 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.946557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.946724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.946802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.946974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.947054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.947243 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.947325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.947504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.947592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.947717 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.947798 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.947967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.948046 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.948212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.948291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.948447 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.948526 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.948679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.948757 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.948906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.948983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.949137 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.949216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.949410 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.949492 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.949665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.949742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.949923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.950052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.950205 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.950319 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.950412 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.950495 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.950640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.950718 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] token.caching = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.950874 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] token.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.950954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.951108 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.951260 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.951374 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.951452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.951607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.951685 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.951860 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.951947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.952101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.952179 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.952334 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.952411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.952576 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.952662 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.952818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.952965 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.953039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.953098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.953287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.953374 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.953676 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.953756 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.953955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.954099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.954244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.954480 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.954569 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.954728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.954999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.955078 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.955291 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.connection = **** {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.955563 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.955673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.955801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.955919 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.956044 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.956289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.956395 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.956530 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.956772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.956859 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.connection = **** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.957019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.957257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.957336 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.957505 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.957688 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.957845 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.957977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.958118 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.958199 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.958452 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.958530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.958692 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.958937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.959016 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.959212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.959374 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.959454 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.959656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.959766 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.959844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 19 11:04:02.960171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.960671 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.960917 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.961167 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.961405 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.961689 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.961929 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.962204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.962439 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.962680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.962953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:04:02.963126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 19 11:04:02.993045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:02.993272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:02.993504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:02.994535 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:02.994762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:02.994982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.002812 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.004581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.039375 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.039678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.040053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.040291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.098470 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.098954 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.148320 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.148557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.149449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.149691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.308156 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.308390 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.308618 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.309465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.309607 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.309758 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.310003 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.310003 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.310127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.310202 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.310396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.310613 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.334284 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.334511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.334743 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.334971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.335115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.335221 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.335388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.335470 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.335622 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.335705 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.335853 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.336078 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.336301 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.336523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.347072 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.348448 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.362678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.362833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.362906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.363064 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.363148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.363325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.363409 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.363551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.363648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.363771 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.363851 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.363990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.364071 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.364206 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:04:03.380922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.381129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 11:04:03.381479 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.381601 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.381684 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 11:04:03.381817 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 11:04:03.381944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.382165 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.382245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.382371 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 11:04:03.382450 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.382656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.382748 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.382925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.383208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.383479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.392847 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.393127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.393404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.393692 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.394075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.394363 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.394642 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.395018 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.400821 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.402171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.404325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.416707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.416707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.416707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.416707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.416707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.416707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.417094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.417164 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.417235 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.417307 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.417388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.417461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.417531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.417601 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.417661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.417661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.417661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.417661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.417661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.417661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.417961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.429654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.431146 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.431463 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.432188 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.433713 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.434012 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.436153 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.436437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.436722 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.438828 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.439117 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.439431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.443101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.443421 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.445876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.446459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.449200 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.450085 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.450381 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.450675 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.450965 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.451277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.451561 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.451833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.452640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.452992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.453282 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.453575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.453864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.454149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.454430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.454706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.467881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.468164 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.470799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.471086 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.472029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.474915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.475280 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.475586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.478150 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.478454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.479781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.482691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.482936 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.486163 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.486262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.486509 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.487493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.487493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.487493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.487812 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.489523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.489832 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.490124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.490424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.490718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.491007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.503702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.503999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.504270 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.504548 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.508473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.508767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.509043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.509315 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.511618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.513781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.516197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.516197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.517006 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.518301 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.518573 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.518677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.518862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.519149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.519568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.520142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.520427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.523109 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.523432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.523722 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.524015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.524416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.530710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.530997 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.531305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.531595 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.531977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.532262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.532543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.535750 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.536041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.536332 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.536686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.537048 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.537334 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.537620 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.548001 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.550902 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.553315 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.553825 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.555368 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.556356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.556836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.557138 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.557626 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.559918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.561375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.562916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.567707 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.567707 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.567853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.567853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.568760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.569160 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.569453 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.569737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.570102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.571615 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.575105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.575527 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.575822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.576107 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.578181 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.578488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.578778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.584192 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.584503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.584800 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.585627 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.585910 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.586196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.586478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.586757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.587034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.587350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.587637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.587912 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.588204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.588491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.588759 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.589041 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.589335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.591747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:04:03.592036 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.592323 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.592605 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.592885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.593165 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.593459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.593732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.594023 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.594299 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.594587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.594976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.595335 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.595608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:04:03.623943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5560484789c0 pid: 81738 (default app) Sep 19 11:04:03.627336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f0c97-9581-42a4-b867-92305ee10820 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:03.627625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f0c97-9581-42a4-b867-92305ee10820 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:03.627900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f0c97-9581-42a4-b867-92305ee10820 None None] PATH_INFO: `/v3` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:03.631664 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5560484789c0 pid: 81737 (default app) Sep 19 11:04:03.632384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1/1] 10.222.0.78 () {50 vars in 791 bytes} [Mon Sep 19 11:04:03 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:11.471897 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2c6ef-62fa-4cb5-b290-781e08ce553e None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:11.472393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2c6ef-62fa-4cb5-b290-781e08ce553e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:11.472393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2c6ef-62fa-4cb5-b290-781e08ce553e None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:11.476724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1/2] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:11.482168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce0c86-9de9-4469-87e7-c200da79f334 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:11.482591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce0c86-9de9-4469-87e7-c200da79f334 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:11.482715 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce0c86-9de9-4469-87e7-c200da79f334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:11.553245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: INFO dbcounter [None req-b9ce0c86-9de9-4469-87e7-c200da79f334 None None] Registered counter for database keystone Sep 19 11:04:11.572124 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG oslo_db.sqlalchemy.engines [None req-b9ce0c86-9de9-4469-87e7-c200da79f334 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81738) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 11:04:11.578882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writer thread running {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 11:04:11.755842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG passlib.handlers.bcrypt [None req-b9ce0c86-9de9-4469-87e7-c200da79f334 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81738) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 19 11:04:11.756235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG passlib.handlers.bcrypt [None req-b9ce0c86-9de9-4469-87e7-c200da79f334 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81738) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 19 11:04:11.814249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-b9ce0c86-9de9-4469-87e7-c200da79f334 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:11.815877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b9ce0c86-9de9-4469-87e7-c200da79f334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:11.944177 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 2/3] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 19 11:04:12.401418 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2c6ef-62fa-4cb5-b290-781e08ce553e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:12.473046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: INFO dbcounter [None req-d9a2c6ef-62fa-4cb5-b290-781e08ce553e None None] Registered counter for database keystone Sep 19 11:04:12.495155 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG oslo_db.sqlalchemy.engines [None req-d9a2c6ef-62fa-4cb5-b290-781e08ce553e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81737) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 11:04:12.501365 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writer thread running {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 11:04:12.626172 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2c6ef-62fa-4cb5-b290-781e08ce553e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:12.630958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd64842-0d32-443f-a448-afcaa1dae921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:12.631619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd64842-0d32-443f-a448-afcaa1dae921 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:12.631909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd64842-0d32-443f-a448-afcaa1dae921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:12.632386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd64842-0d32-443f-a448-afcaa1dae921 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:12.642545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd64842-0d32-443f-a448-afcaa1dae921 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:12.695161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: WARNING keystone.common.rbac_enforcer.enforcer [None req-dbd64842-0d32-443f-a448-afcaa1dae921 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 19 11:04:12.695478 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd64842-0d32-443f-a448-afcaa1dae921 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:12.705721 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 2/4] 10.222.0.78 () {58 vars in 1324 bytes} [Mon Sep 19 11:04:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:04:12.711835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce0c86-9de9-4469-87e7-c200da79f334 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:12.728912 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce0c86-9de9-4469-87e7-c200da79f334 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:12.733947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0c9b1-97cc-4d51-bdc7-67855637d6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:12.735053 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0c9b1-97cc-4d51-bdc7-67855637d6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:12.735389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0c9b1-97cc-4d51-bdc7-67855637d6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:12.735655 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0c9b1-97cc-4d51-bdc7-67855637d6d5 admin admin] PATH_INFO: `/v3/projects/a875b13c6f2b4a43a97ce59e85aa99fd` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:12.737217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde0c9b1-97cc-4d51-bdc7-67855637d6d5 admin admin] RBAC: Authorizing `identity:get_project(project_id=a875b13c6f2b4a43a97ce59e85aa99fd)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:12.789807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cde0c9b1-97cc-4d51-bdc7-67855637d6d5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 19 11:04:12.790145 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde0c9b1-97cc-4d51-bdc7-67855637d6d5 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:12.795340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 3/5] 10.222.0.78 () {56 vars in 1207 bytes} [Mon Sep 19 11:04:12 2022] GET /identity/v3/projects/a875b13c6f2b4a43a97ce59e85aa99fd => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:14.953699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc0a8c-5692-465d-9cb5-75055614eb44 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:14.954215 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc0a8c-5692-465d-9cb5-75055614eb44 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:14.954295 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc0a8c-5692-465d-9cb5-75055614eb44 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:14.958349 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 3/6] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:14.963785 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cab61-12f9-4feb-babe-ad8daba5111e None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:14.964190 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cab61-12f9-4feb-babe-ad8daba5111e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:14.964338 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cab61-12f9-4feb-babe-ad8daba5111e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:14.983974 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-c43cab61-12f9-4feb-babe-ad8daba5111e None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:14.985489 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c43cab61-12f9-4feb-babe-ad8daba5111e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:14.993074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 4/7] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 19 11:04:15.261459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc0a8c-5692-465d-9cb5-75055614eb44 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:15.273894 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc0a8c-5692-465d-9cb5-75055614eb44 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:15.278316 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8a9dd-adcd-4030-8c85-39700d113b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:15.278921 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8a9dd-adcd-4030-8c85-39700d113b20 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:15.279233 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8a9dd-adcd-4030-8c85-39700d113b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:15.279509 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8a9dd-adcd-4030-8c85-39700d113b20 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:15.288722 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8a9dd-adcd-4030-8c85-39700d113b20 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:15.290510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8a9dd-adcd-4030-8c85-39700d113b20 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:15.300376 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 4/8] 10.222.0.78 () {58 vars in 1324 bytes} [Mon Sep 19 11:04:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:04:15.306899 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cab61-12f9-4feb-babe-ad8daba5111e None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:15.317979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cab61-12f9-4feb-babe-ad8daba5111e None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:15.323685 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9332931-d616-48f5-922c-26da058cbc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:15.324327 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9332931-d616-48f5-922c-26da058cbc5b admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:15.324610 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9332931-d616-48f5-922c-26da058cbc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:15.324885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9332931-d616-48f5-922c-26da058cbc5b admin admin] PATH_INFO: `/v3/users/b83cd9cf7ce84815942f0d89b245aefe` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:15.326209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9332931-d616-48f5-922c-26da058cbc5b admin admin] RBAC: Authorizing `identity:get_user(user_id=b83cd9cf7ce84815942f0d89b245aefe)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:15.328089 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9332931-d616-48f5-922c-26da058cbc5b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:15.329646 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 5/9] 10.222.0.78 () {56 vars in 1198 bytes} [Mon Sep 19 11:04:15 2022] GET /identity/v3/users/b83cd9cf7ce84815942f0d89b245aefe => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:17.589046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c7e58-99dc-4ccf-be07-1cf9c1d24d87 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:17.590228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c7e58-99dc-4ccf-be07-1cf9c1d24d87 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:17.590228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c7e58-99dc-4ccf-be07-1cf9c1d24d87 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:17.591505 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 5/10] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:04:17.595114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e1a08-a2e9-4d3f-ac29-d08f4598621b None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:17.595975 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e1a08-a2e9-4d3f-ac29-d08f4598621b None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:17.596263 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e1a08-a2e9-4d3f-ac29-d08f4598621b None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:17.596711 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb44ed-0f58-4253-8617-34bbf97d33a4 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:17.596992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb44ed-0f58-4253-8617-34bbf97d33a4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:17.597261 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb44ed-0f58-4253-8617-34bbf97d33a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:17.597594 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 6/11] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:04:17.602743 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705a70a-24e2-413a-9e44-a255e771dba6 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:17.603033 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705a70a-24e2-413a-9e44-a255e771dba6 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:17.603350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705a70a-24e2-413a-9e44-a255e771dba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:17.621171 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-8705a70a-24e2-413a-9e44-a255e771dba6 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:17.622368 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-8705a70a-24e2-413a-9e44-a255e771dba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:17.640695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG passlib.handlers.bcrypt [None req-4dfb44ed-0f58-4253-8617-34bbf97d33a4 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81737) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 19 11:04:17.641082 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG passlib.handlers.bcrypt [None req-4dfb44ed-0f58-4253-8617-34bbf97d33a4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81737) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 19 11:04:17.689914 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4dfb44ed-0f58-4253-8617-34bbf97d33a4 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:17.691604 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4dfb44ed-0f58-4253-8617-34bbf97d33a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:17.703944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 7/12] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:04:17.791021 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 6/13] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:04:17.971391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705a70a-24e2-413a-9e44-a255e771dba6 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:17.997212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:18.053005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb44ed-0f58-4253-8617-34bbf97d33a4 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:18.078248 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705a70a-24e2-413a-9e44-a255e771dba6 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:18.115158 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d25a6-9a81-459f-97b4-0ef1395a5fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:18.115845 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d25a6-9a81-459f-97b4-0ef1395a5fdb None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:18.116123 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d25a6-9a81-459f-97b4-0ef1395a5fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:18.116397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d25a6-9a81-459f-97b4-0ef1395a5fdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:18.128661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d25a6-9a81-459f-97b4-0ef1395a5fdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:18.137559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-d23d25a6-9a81-459f-97b4-0ef1395a5fdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:04:18.137559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:18.137559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:18.137559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:18.137559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:18.137559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:18.137559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:18.137559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:18.137559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:18.137559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:18.137559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:18.137559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:18.137559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:18.137559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:18.137559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:18.137559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:18.138676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:18.138676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:18.138676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:18.138676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:18.138676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:18.138676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:18.138676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:18.138676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:18.138676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:18.138676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:18.138676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:18.138676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:18.138676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:18.138676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:18.138676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:18.138676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:18.139819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:18.139819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:18.139819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:18.139819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:18.139819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:18.139819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:18.139819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:18.139819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:18.139819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:18.139819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:18.139819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:04:18.139819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:04:18.139819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 8/14] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:04:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:18.144419 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d25a6-9a81-459f-97b4-0ef1395a5fdb None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:18.161329 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb44ed-0f58-4253-8617-34bbf97d33a4 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:18.163754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:18.198921 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af4a91-c8e3-4478-953c-29ad6519518e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:18.199822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af4a91-c8e3-4478-953c-29ad6519518e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:18.200112 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af4a91-c8e3-4478-953c-29ad6519518e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:18.200393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af4a91-c8e3-4478-953c-29ad6519518e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:18.228396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:18.246700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d25a6-9a81-459f-97b4-0ef1395a5fdb None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:18.267957 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af4a91-c8e3-4478-953c-29ad6519518e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:18.269486 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af4a91-c8e3-4478-953c-29ad6519518e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:18.280584 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f71f5-faf5-4734-9f5e-1517e859c90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:18.281189 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f71f5-faf5-4734-9f5e-1517e859c90e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:18.281473 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f71f5-faf5-4734-9f5e-1517e859c90e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:18.281745 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f71f5-faf5-4734-9f5e-1517e859c90e None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:18.282408 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f71f5-faf5-4734-9f5e-1517e859c90e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:18.282720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f71f5-faf5-4734-9f5e-1517e859c90e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:18.284139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f71f5-faf5-4734-9f5e-1517e859c90e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:18.296340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 9/15] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:18.302178 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f71f5-faf5-4734-9f5e-1517e859c90e None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:18.369676 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 7/16] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:04:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 11:04:18.375994 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af4a91-c8e3-4478-953c-29ad6519518e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:18.378887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:18.403193 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f71f5-faf5-4734-9f5e-1517e859c90e None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:18.442054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446dd990-4fb8-4149-bb86-c0d0f3137f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:18.442691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dd990-4fb8-4149-bb86-c0d0f3137f02 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:18.442973 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dd990-4fb8-4149-bb86-c0d0f3137f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:18.443273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dd990-4fb8-4149-bb86-c0d0f3137f02 None admin] PATH_INFO: `/v3/users/b83cd9cf7ce84815942f0d89b245aefe` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:18.444431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446dd990-4fb8-4149-bb86-c0d0f3137f02 None admin] RBAC: Authorizing `identity:get_user(user_id=b83cd9cf7ce84815942f0d89b245aefe)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:18.445898 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446dd990-4fb8-4149-bb86-c0d0f3137f02 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:18.447524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 10/17] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:18 2022] GET /identity/v3/users/b83cd9cf7ce84815942f0d89b245aefe => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:18.453024 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446dd990-4fb8-4149-bb86-c0d0f3137f02 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:18.468083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:18.495907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af4a91-c8e3-4478-953c-29ad6519518e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:18.530686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07ff52-bdef-47ff-af98-96b14f34e507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:18.531575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07ff52-bdef-47ff-af98-96b14f34e507 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:18.531851 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07ff52-bdef-47ff-af98-96b14f34e507 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:18.532140 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07ff52-bdef-47ff-af98-96b14f34e507 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:18.551067 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07ff52-bdef-47ff-af98-96b14f34e507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:18.552596 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07ff52-bdef-47ff-af98-96b14f34e507 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:18.559928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446dd990-4fb8-4149-bb86-c0d0f3137f02 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:18.572195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-cd07ff52-bdef-47ff-af98-96b14f34e507 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:04:18.572195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:18.572195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:18.572195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:18.572195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:18.572195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:18.572195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:18.572195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:18.572195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:18.572195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:18.572195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:18.572195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:18.572195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:04:18.572195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:04:18.572195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:04:18.572195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:04:18.573201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:18.573201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:18.573201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:18.573201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:18.573201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:18.573201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:18.573201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:18.573201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:18.573201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:18.573201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:18.573201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:18.573201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:18.573201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:18.573201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:18.573201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:18.573201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:18.573201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:04:18.574075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:04:18.574075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:04:18.574075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:04:18.574075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 8/18] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:04:18.579148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07ff52-bdef-47ff-af98-96b14f34e507 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:18.595736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d7cd7-8d36-4454-993c-c558d2ecd197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:18.596373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d7cd7-8d36-4454-993c-c558d2ecd197 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:18.596656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d7cd7-8d36-4454-993c-c558d2ecd197 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:18.596932 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d7cd7-8d36-4454-993c-c558d2ecd197 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:18.598062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d7cd7-8d36-4454-993c-c558d2ecd197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:18.599549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d7cd7-8d36-4454-993c-c558d2ecd197 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:18.601104 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 11/19] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:04:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:18.607429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d7cd7-8d36-4454-993c-c558d2ecd197 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:18.636792 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:18.702959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07ff52-bdef-47ff-af98-96b14f34e507 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:18.718687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:18.740990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d7cd7-8d36-4454-993c-c558d2ecd197 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:18.747673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41cdf27-3e84-4222-a072-d19195e9285a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:18.748240 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cdf27-3e84-4222-a072-d19195e9285a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:18.748530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cdf27-3e84-4222-a072-d19195e9285a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:18.748804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cdf27-3e84-4222-a072-d19195e9285a None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:18.749464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41cdf27-3e84-4222-a072-d19195e9285a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:18.749765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41cdf27-3e84-4222-a072-d19195e9285a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:18.751207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41cdf27-3e84-4222-a072-d19195e9285a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:18.768948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 9/20] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:18.786970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1d499-8e08-4ea5-9c11-f2986b31ec93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:18.787640 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1d499-8e08-4ea5-9c11-f2986b31ec93 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:18.787920 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1d499-8e08-4ea5-9c11-f2986b31ec93 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:18.788192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1d499-8e08-4ea5-9c11-f2986b31ec93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:18.789332 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1d499-8e08-4ea5-9c11-f2986b31ec93 None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.domain.id=default, user.id=b83cd9cf7ce84815942f0d89b245aefe) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:18.789332 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1d499-8e08-4ea5-9c11-f2986b31ec93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:18.790750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1d499-8e08-4ea5-9c11-f2986b31ec93 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:18.799322 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 12/21] 10.222.0.78 () {56 vars in 1322 bytes} [Mon Sep 19 11:04:18 2022] GET /identity/v3/role_assignments?user.id=b83cd9cf7ce84815942f0d89b245aefe&scope.domain.id=default&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 233 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:21.254087 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c637e4-0676-438f-9e70-2cf26de022fe None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:21.255496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c637e4-0676-438f-9e70-2cf26de022fe None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:21.256024 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c637e4-0676-438f-9e70-2cf26de022fe None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:21.258119 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 10/22] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:21.273038 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e44d0d-5e86-4790-8bef-053dfa7428f8 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:21.273970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e44d0d-5e86-4790-8bef-053dfa7428f8 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:21.274580 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e44d0d-5e86-4790-8bef-053dfa7428f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:21.304370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-32e44d0d-5e86-4790-8bef-053dfa7428f8 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:21.305950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-32e44d0d-5e86-4790-8bef-053dfa7428f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:21.403882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 13/23] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:21.450940 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28832c-4f75-4951-937d-1481749b01be None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:21.451661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28832c-4f75-4951-937d-1481749b01be None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:21.452284 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28832c-4f75-4951-937d-1481749b01be None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:21.454013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 11/24] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:21.460010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170dee0-c688-4198-b652-dccb20cf036c None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:21.460670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170dee0-c688-4198-b652-dccb20cf036c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:21.461162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170dee0-c688-4198-b652-dccb20cf036c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:21.492243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-4170dee0-c688-4198-b652-dccb20cf036c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:21.494035 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4170dee0-c688-4198-b652-dccb20cf036c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:21.593669 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 14/25] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:21.805825 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28832c-4f75-4951-937d-1481749b01be None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:21.815945 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab7750-3c00-444a-96e2-7486b1d19035 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:21.816548 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab7750-3c00-444a-96e2-7486b1d19035 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:21.817045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab7750-3c00-444a-96e2-7486b1d19035 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:21.818773 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 15/26] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:21.826249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1b890-6231-4a7e-b123-6ccbf6c7feeb None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:21.826920 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1b890-6231-4a7e-b123-6ccbf6c7feeb None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:21.827592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1b890-6231-4a7e-b123-6ccbf6c7feeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:21.840483 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:21.860410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-9ed1b890-6231-4a7e-b123-6ccbf6c7feeb None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:21.861965 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-9ed1b890-6231-4a7e-b123-6ccbf6c7feeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:21.989413 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28832c-4f75-4951-937d-1481749b01be None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:22.009841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 16/27] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:04:22.011581 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded06200-68f5-4fe2-a4e4-58850e7c5996 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:22.012016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded06200-68f5-4fe2-a4e4-58850e7c5996 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:22.012456 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded06200-68f5-4fe2-a4e4-58850e7c5996 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:22.013164 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:22.014312 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 17/28] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:22.015196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded06200-68f5-4fe2-a4e4-58850e7c5996 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:22.048278 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b3f65-bfde-427b-8093-86de3762b1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:22.050682 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b3f65-bfde-427b-8093-86de3762b1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:22.051196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b3f65-bfde-427b-8093-86de3762b1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:22.052679 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b3f65-bfde-427b-8093-86de3762b1b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:22.076833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b3f65-bfde-427b-8093-86de3762b1b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:22.089098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-6f6b3f65-bfde-427b-8093-86de3762b1b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:04:22.089098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:22.089098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:22.089098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:22.089098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:22.089098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:22.089098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:22.089098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:22.089098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:22.089098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:22.089098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:22.089098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:22.089098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:22.089098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:22.089098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:22.089098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:22.090169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:22.090169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:22.090169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:22.090169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:22.090169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:22.090169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:22.090169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:22.090169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:22.090169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:22.090169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:22.090169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:22.090169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:22.090169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:22.090169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:22.090169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:22.090169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:22.091217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:22.091217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:22.091217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:22.091217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:22.091217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:22.091217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:22.091217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:22.091217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:22.091217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:22.091217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:22.091217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:04:22.091217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:04:22.100277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 12/29] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:04:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:22.101708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb739fcb-1204-4908-a18f-36ab83be1b84 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:22.102284 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb739fcb-1204-4908-a18f-36ab83be1b84 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:22.105345 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb739fcb-1204-4908-a18f-36ab83be1b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:22.144636 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-bb739fcb-1204-4908-a18f-36ab83be1b84 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:22.145869 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bb739fcb-1204-4908-a18f-36ab83be1b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:22.239891 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded06200-68f5-4fe2-a4e4-58850e7c5996 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:22.243676 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 13/30] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:22.245617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb739fcb-1204-4908-a18f-36ab83be1b84 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:22.312887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aaa33-0219-4b92-9b06-5eeea70fb64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:22.313575 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aaa33-0219-4b92-9b06-5eeea70fb64e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:22.313816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aaa33-0219-4b92-9b06-5eeea70fb64e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:22.314092 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aaa33-0219-4b92-9b06-5eeea70fb64e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:22.319368 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:22.332559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402aaa33-0219-4b92-9b06-5eeea70fb64e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:22.333982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402aaa33-0219-4b92-9b06-5eeea70fb64e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:22.374755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-402aaa33-0219-4b92-9b06-5eeea70fb64e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:04:22.374755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:22.374755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:22.374755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:22.374755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:22.374755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:22.374755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:22.374755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:22.374755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:22.374755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:22.374755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:22.374755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:22.374755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:04:22.374755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:04:22.374755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:04:22.374755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:04:22.375751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:22.375751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:22.375751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:22.375751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:22.375751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:22.375751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:22.375751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:22.375751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:22.375751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:22.375751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:22.375751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:22.375751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:22.375751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:22.375751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:22.375751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:22.375751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:22.375751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:04:22.376562 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:04:22.376562 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:04:22.376562 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:04:22.384433 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 18/31] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:04:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:22.385271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aaa33-0219-4b92-9b06-5eeea70fb64e None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:22.386690 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb739fcb-1204-4908-a18f-36ab83be1b84 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:22.404882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:22.457096 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743015b1-a858-4478-9d5d-34f073ca68dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:22.457716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743015b1-a858-4478-9d5d-34f073ca68dc None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:22.457994 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743015b1-a858-4478-9d5d-34f073ca68dc None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:22.458267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743015b1-a858-4478-9d5d-34f073ca68dc None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:22.458934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743015b1-a858-4478-9d5d-34f073ca68dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:22.459256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743015b1-a858-4478-9d5d-34f073ca68dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:22.460641 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743015b1-a858-4478-9d5d-34f073ca68dc None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:22.487265 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 14/32] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:22.489518 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743015b1-a858-4478-9d5d-34f073ca68dc None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:22.535710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aaa33-0219-4b92-9b06-5eeea70fb64e None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:22.594788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c45bc1-f834-40fe-8105-d0fbe0eb11f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:22.595345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c45bc1-f834-40fe-8105-d0fbe0eb11f4 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:22.595626 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c45bc1-f834-40fe-8105-d0fbe0eb11f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:22.595896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c45bc1-f834-40fe-8105-d0fbe0eb11f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:22.596564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c45bc1-f834-40fe-8105-d0fbe0eb11f4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:22.597960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c45bc1-f834-40fe-8105-d0fbe0eb11f4 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:22.649496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743015b1-a858-4478-9d5d-34f073ca68dc None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:22.686778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cdc55-eb70-4b08-a97d-a9b1f36f3656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:22.687432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cdc55-eb70-4b08-a97d-a9b1f36f3656 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:22.687715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cdc55-eb70-4b08-a97d-a9b1f36f3656 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:22.687990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cdc55-eb70-4b08-a97d-a9b1f36f3656 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:22.688646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668cdc55-eb70-4b08-a97d-a9b1f36f3656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:22.688949 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668cdc55-eb70-4b08-a97d-a9b1f36f3656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:22.690355 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668cdc55-eb70-4b08-a97d-a9b1f36f3656 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:22.702285 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 15/33] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:22.703216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cdc55-eb70-4b08-a97d-a9b1f36f3656 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:22.734235 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:22.812421 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cdc55-eb70-4b08-a97d-a9b1f36f3656 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:22.857149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9072fd-b3a4-4d27-965d-290e40ea2e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:22.858213 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9072fd-b3a4-4d27-965d-290e40ea2e71 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:22.858691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9072fd-b3a4-4d27-965d-290e40ea2e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:22.859159 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9072fd-b3a4-4d27-965d-290e40ea2e71 None admin] PATH_INFO: `/v3/users/b83cd9cf7ce84815942f0d89b245aefe` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:22.861197 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9072fd-b3a4-4d27-965d-290e40ea2e71 None admin] RBAC: Authorizing `identity:get_user(user_id=b83cd9cf7ce84815942f0d89b245aefe)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:22.862914 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9072fd-b3a4-4d27-965d-290e40ea2e71 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:22.864878 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 16/34] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:22 2022] GET /identity/v3/users/b83cd9cf7ce84815942f0d89b245aefe => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:22.865809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9072fd-b3a4-4d27-965d-290e40ea2e71 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:22.868988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 19/35] 10.222.0.78 () {60 vars in 1131 bytes} [Mon Sep 19 11:04:22 2022] POST /identity/v3/roles => generated 225 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:04:22.871299 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c45bc1-f834-40fe-8105-d0fbe0eb11f4 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:22.924768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:22.972692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:22.979564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9072fd-b3a4-4d27-965d-290e40ea2e71 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:22.986102 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c45bc1-f834-40fe-8105-d0fbe0eb11f4 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:23.017993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd8683-739e-473e-bbba-ff2724faa7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:23.018615 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd8683-739e-473e-bbba-ff2724faa7e3 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:23.018898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd8683-739e-473e-bbba-ff2724faa7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:23.019201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd8683-739e-473e-bbba-ff2724faa7e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:23.023023 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd8683-739e-473e-bbba-ff2724faa7e3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:23.023023 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd8683-739e-473e-bbba-ff2724faa7e3 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:23.023207 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb6c7a-2094-415b-a102-eb3e793a7904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:23.023748 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb6c7a-2094-415b-a102-eb3e793a7904 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:23.024031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb6c7a-2094-415b-a102-eb3e793a7904 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:23.024557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb6c7a-2094-415b-a102-eb3e793a7904 None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:23.025228 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb6c7a-2094-415b-a102-eb3e793a7904 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:23.027400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb6c7a-2094-415b-a102-eb3e793a7904 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:23.135345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 20/36] 10.222.0.78 () {60 vars in 1141 bytes} [Mon Sep 19 11:04:22 2022] POST /identity/v3/projects => generated 301 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:04:23.135345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb6c7a-2094-415b-a102-eb3e793a7904 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:23.218865 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 17/37] 10.222.0.78 () {60 vars in 1131 bytes} [Mon Sep 19 11:04:22 2022] POST /identity/v3/roles => generated 219 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:04:23.220431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b210240-9ff6-493e-a112-7843b4e0ce9a None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:23.220724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b210240-9ff6-493e-a112-7843b4e0ce9a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:23.221007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b210240-9ff6-493e-a112-7843b4e0ce9a None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:23.227217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 18/38] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:23.229903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17622f6-322d-4da3-95f9-78278d47b809 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:23.230712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17622f6-322d-4da3-95f9-78278d47b809 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:23.230982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17622f6-322d-4da3-95f9-78278d47b809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:23.253026 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a17622f6-322d-4da3-95f9-78278d47b809 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:23.254794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a17622f6-322d-4da3-95f9-78278d47b809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:23.297254 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:23.302673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb6c7a-2094-415b-a102-eb3e793a7904 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:23.364195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 19/39] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:04:23.376163 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70d2ec-af6f-4c8b-b569-c7775639c4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:23.377504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d2ec-af6f-4c8b-b569-c7775639c4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:23.378016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d2ec-af6f-4c8b-b569-c7775639c4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:23.378490 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d2ec-af6f-4c8b-b569-c7775639c4f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:23.380356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70d2ec-af6f-4c8b-b569-c7775639c4f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:23.382158 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70d2ec-af6f-4c8b-b569-c7775639c4f7 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:23.384112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 21/40] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:04:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:23.390827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17622f6-322d-4da3-95f9-78278d47b809 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:23.457373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce64fd-1b0e-4eac-ace2-22879e597fd7 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:23.458024 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce64fd-1b0e-4eac-ace2-22879e597fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:23.458407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce64fd-1b0e-4eac-ace2-22879e597fd7 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:23.459944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 22/41] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:04:23.466681 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b1e25-6cde-4254-af2d-4f1a4284089a None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:23.466681 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b1e25-6cde-4254-af2d-4f1a4284089a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:23.467052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b1e25-6cde-4254-af2d-4f1a4284089a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:23.507602 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:23.527333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17622f6-322d-4da3-95f9-78278d47b809 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:23.569698 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ae800-6ff6-497c-8bc3-ebfbfbf5e1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:23.570765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ae800-6ff6-497c-8bc3-ebfbfbf5e1a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:23.571270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ae800-6ff6-497c-8bc3-ebfbfbf5e1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:23.571873 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ae800-6ff6-497c-8bc3-ebfbfbf5e1a4 None admin] PATH_INFO: `/v3/domains/default/users/b83cd9cf7ce84815942f0d89b245aefe/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:23.575104 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ae800-6ff6-497c-8bc3-ebfbfbf5e1a4 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b83cd9cf7ce84815942f0d89b245aefe, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:23.577162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ae800-6ff6-497c-8bc3-ebfbfbf5e1a4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:23.603612 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-481b1e25-6cde-4254-af2d-4f1a4284089a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:23.605340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-481b1e25-6cde-4254-af2d-4f1a4284089a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:23.676409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 23/42] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:04:23.677170 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b1e25-6cde-4254-af2d-4f1a4284089a None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:23.699750 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 20/43] 10.222.0.78 () {58 vars in 1361 bytes} [Mon Sep 19 11:04:23 2022] PUT /identity/v3/domains/default/users/b83cd9cf7ce84815942f0d89b245aefe/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:04:23.785612 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:23.794062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b1e25-6cde-4254-af2d-4f1a4284089a None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:23.834399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca16e0-b7a4-4bdb-9020-8161363f27b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:23.835046 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca16e0-b7a4-4bdb-9020-8161363f27b7 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:23.835363 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca16e0-b7a4-4bdb-9020-8161363f27b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:23.835647 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca16e0-b7a4-4bdb-9020-8161363f27b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:23.836337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca16e0-b7a4-4bdb-9020-8161363f27b7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:23.837741 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca16e0-b7a4-4bdb-9020-8161363f27b7 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:23.899566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 24/44] 10.222.0.78 () {60 vars in 1131 bytes} [Mon Sep 19 11:04:23 2022] POST /identity/v3/roles => generated 223 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:04:23.943553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ae800-6ff6-497c-8bc3-ebfbfbf5e1a4 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:24.059503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ae800-6ff6-497c-8bc3-ebfbfbf5e1a4 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:24.099109 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b71be-1917-4eac-8167-bb88286c9763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:24.099681 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b71be-1917-4eac-8167-bb88286c9763 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:24.099969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b71be-1917-4eac-8167-bb88286c9763 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:24.100248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b71be-1917-4eac-8167-bb88286c9763 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:24.101602 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b71be-1917-4eac-8167-bb88286c9763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:24.103067 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b71be-1917-4eac-8167-bb88286c9763 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:24.105379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 21/45] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:04:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:24.114673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca16e0-b7a4-4bdb-9020-8161363f27b7 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:24.219046 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca16e0-b7a4-4bdb-9020-8161363f27b7 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:24.253048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1ec32-164c-4cf0-ac40-a00583e23be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:24.253739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1ec32-164c-4cf0-ac40-a00583e23be5 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:24.253956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1ec32-164c-4cf0-ac40-a00583e23be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:24.254239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1ec32-164c-4cf0-ac40-a00583e23be5 None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:24.255152 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1ec32-164c-4cf0-ac40-a00583e23be5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:24.256626 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1ec32-164c-4cf0-ac40-a00583e23be5 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:24.363953 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 25/46] 10.222.0.78 () {60 vars in 1141 bytes} [Mon Sep 19 11:04:24 2022] POST /identity/v3/projects => generated 312 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:04:25.898483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad226799-715e-46bc-a6df-943b1f7e6d5b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:25.899006 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad226799-715e-46bc-a6df-943b1f7e6d5b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:25.899086 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad226799-715e-46bc-a6df-943b1f7e6d5b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:25.900410 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 22/47] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:25.906342 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa153877-3225-4528-baff-8fe564b949c2 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:25.906723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa153877-3225-4528-baff-8fe564b949c2 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:25.907013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa153877-3225-4528-baff-8fe564b949c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:25.925126 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-fa153877-3225-4528-baff-8fe564b949c2 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:25.926317 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-fa153877-3225-4528-baff-8fe564b949c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:25.928362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:25.993377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 26/48] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:04:26.266407 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad226799-715e-46bc-a6df-943b1f7e6d5b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:26.282538 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:26.367313 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad226799-715e-46bc-a6df-943b1f7e6d5b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:26.401094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f98422-5a55-4b19-8acf-3f8acb6936e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:26.401745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f98422-5a55-4b19-8acf-3f8acb6936e8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:26.402027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f98422-5a55-4b19-8acf-3f8acb6936e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:26.402298 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f98422-5a55-4b19-8acf-3f8acb6936e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:26.413069 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f98422-5a55-4b19-8acf-3f8acb6936e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:26.418613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-11f98422-5a55-4b19-8acf-3f8acb6936e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:04:26.418613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:26.418613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:26.418613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:26.418613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:26.418613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:26.418613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:26.418613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:26.418613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:26.418613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:26.418613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:26.418613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:26.418613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:26.418613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:26.418613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:26.418613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:26.419718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:26.419718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:26.419718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:26.419718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:26.419718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:26.419718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:26.419718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:26.419718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:26.419718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:26.419718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:26.419718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:26.419718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:26.419718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:26.419718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:26.419718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:26.419718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:26.420520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:26.420520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:26.420520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:26.420520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:26.420520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:26.420520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:26.420520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:26.420520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:26.420520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:26.420520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:26.420520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:04:26.420520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:04:26.420520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 23/49] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:04:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:26.426767 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa153877-3225-4528-baff-8fe564b949c2 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:26.528196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa153877-3225-4528-baff-8fe564b949c2 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:26.554936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5a5d3-a299-4930-bc03-69a4153eb6f3 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:26.555253 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5a5d3-a299-4930-bc03-69a4153eb6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:26.555533 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5a5d3-a299-4930-bc03-69a4153eb6f3 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:26.556853 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 24/50] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:04:26.562828 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1682d5-3ba1-4e5d-9d60-db570b1140ca None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:26.563117 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1682d5-3ba1-4e5d-9d60-db570b1140ca None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:26.563429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1682d5-3ba1-4e5d-9d60-db570b1140ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:26.563614 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fdac8-6c92-4fed-9c71-d068cfaaf64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:26.564239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fdac8-6c92-4fed-9c71-d068cfaaf64e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:26.564547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fdac8-6c92-4fed-9c71-d068cfaaf64e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:26.564874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fdac8-6c92-4fed-9c71-d068cfaaf64e None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:26.565574 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fdac8-6c92-4fed-9c71-d068cfaaf64e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:26.565860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fdac8-6c92-4fed-9c71-d068cfaaf64e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:26.567287 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fdac8-6c92-4fed-9c71-d068cfaaf64e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:26.572836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:26.576417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 27/51] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:26.582431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fdac8-6c92-4fed-9c71-d068cfaaf64e None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:26.685855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-fa1682d5-3ba1-4e5d-9d60-db570b1140ca None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:26.687128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fa1682d5-3ba1-4e5d-9d60-db570b1140ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:26.692854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fdac8-6c92-4fed-9c71-d068cfaaf64e None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:26.729295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c0f42-a521-447a-9acc-110787bf488a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:26.730526 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c0f42-a521-447a-9acc-110787bf488a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:26.730807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c0f42-a521-447a-9acc-110787bf488a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:26.731154 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c0f42-a521-447a-9acc-110787bf488a None admin] PATH_INFO: `/v3/users/b83cd9cf7ce84815942f0d89b245aefe` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:26.732301 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c0f42-a521-447a-9acc-110787bf488a None admin] RBAC: Authorizing `identity:get_user(user_id=b83cd9cf7ce84815942f0d89b245aefe)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:26.733762 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c0f42-a521-447a-9acc-110787bf488a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:26.735472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 28/52] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:26 2022] GET /identity/v3/users/b83cd9cf7ce84815942f0d89b245aefe => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:26.741670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c0f42-a521-447a-9acc-110787bf488a None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:26.760579 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 25/53] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:04:26.811698 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:26.844356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c0f42-a521-447a-9acc-110787bf488a None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:26.877217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b9056-7c1a-46f0-83ed-fd266013dc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:26.877848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b9056-7c1a-46f0-83ed-fd266013dc4c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:26.878135 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b9056-7c1a-46f0-83ed-fd266013dc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:26.878413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b9056-7c1a-46f0-83ed-fd266013dc4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:26.879575 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633b9056-7c1a-46f0-83ed-fd266013dc4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:26.881370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633b9056-7c1a-46f0-83ed-fd266013dc4c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:26.882964 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 29/54] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:04:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:26.888977 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1682d5-3ba1-4e5d-9d60-db570b1140ca None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:26.892527 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:26.987703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1682d5-3ba1-4e5d-9d60-db570b1140ca None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:27.020484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2f245-9977-41ff-bdd5-72c727c7c3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:27.021107 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2f245-9977-41ff-bdd5-72c727c7c3fb None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:27.021395 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2f245-9977-41ff-bdd5-72c727c7c3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:27.021670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2f245-9977-41ff-bdd5-72c727c7c3fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:27.022399 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2f245-9977-41ff-bdd5-72c727c7c3fb None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.domain.id=default, user.id=b83cd9cf7ce84815942f0d89b245aefe) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:27.022712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2f245-9977-41ff-bdd5-72c727c7c3fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:27.024285 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2f245-9977-41ff-bdd5-72c727c7c3fb None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:27.024466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b9056-7c1a-46f0-83ed-fd266013dc4c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:27.034194 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 26/55] 10.222.0.78 () {56 vars in 1322 bytes} [Mon Sep 19 11:04:26 2022] GET /identity/v3/role_assignments?user.id=b83cd9cf7ce84815942f0d89b245aefe&scope.domain.id=default&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:27.138857 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b9056-7c1a-46f0-83ed-fd266013dc4c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:27.175227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd30a2-238f-49e0-9edb-49a4092e591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:27.175913 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd30a2-238f-49e0-9edb-49a4092e591c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:27.176202 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd30a2-238f-49e0-9edb-49a4092e591c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:27.176485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd30a2-238f-49e0-9edb-49a4092e591c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:27.177654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd30a2-238f-49e0-9edb-49a4092e591c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:27.179149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd30a2-238f-49e0-9edb-49a4092e591c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:27.180926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 30/56] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:04:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:27.187008 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2f245-9977-41ff-bdd5-72c727c7c3fb None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:27.265678 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:27.304616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2f245-9977-41ff-bdd5-72c727c7c3fb None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:27.337107 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20d3ad-9bf6-4209-9daa-87e8c9b5ccf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:27.337721 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20d3ad-9bf6-4209-9daa-87e8c9b5ccf7 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:27.338013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20d3ad-9bf6-4209-9daa-87e8c9b5ccf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:27.338290 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20d3ad-9bf6-4209-9daa-87e8c9b5ccf7 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:27.338943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20d3ad-9bf6-4209-9daa-87e8c9b5ccf7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:27.340985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20d3ad-9bf6-4209-9daa-87e8c9b5ccf7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:27.425862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 27/57] 10.222.0.78 () {60 vars in 1140 bytes} [Mon Sep 19 11:04:27 2022] POST /identity/v3/projects => generated 298 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:04:29.566239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d5778-f620-4a63-970e-8173488ceb46 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:29.566778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d5778-f620-4a63-970e-8173488ceb46 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:29.566778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d5778-f620-4a63-970e-8173488ceb46 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:29.568223 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 31/58] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:29.574132 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d95e46-f08c-4ee6-b910-0121ac7a46e3 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:29.574528 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d95e46-f08c-4ee6-b910-0121ac7a46e3 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:29.574701 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d95e46-f08c-4ee6-b910-0121ac7a46e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:29.593173 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-09d95e46-f08c-4ee6-b910-0121ac7a46e3 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:29.594364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-09d95e46-f08c-4ee6-b910-0121ac7a46e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:29.658766 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 28/59] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:29.927921 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d5778-f620-4a63-970e-8173488ceb46 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:29.931860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:30.028540 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d5778-f620-4a63-970e-8173488ceb46 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:30.060626 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225aadd-1af6-4344-9bd4-072e2b67f1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:30.061258 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225aadd-1af6-4344-9bd4-072e2b67f1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:30.061537 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225aadd-1af6-4344-9bd4-072e2b67f1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:30.061815 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225aadd-1af6-4344-9bd4-072e2b67f1a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:30.062935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225aadd-1af6-4344-9bd4-072e2b67f1a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:30.064417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225aadd-1af6-4344-9bd4-072e2b67f1a8 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:30.065983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 32/60] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:04:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:30.071722 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d95e46-f08c-4ee6-b910-0121ac7a46e3 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:30.121877 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:30.170532 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d95e46-f08c-4ee6-b910-0121ac7a46e3 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:30.209080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689aae3e-fe37-4577-a246-319f2bd998a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:30.209615 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689aae3e-fe37-4577-a246-319f2bd998a2 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:30.209903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689aae3e-fe37-4577-a246-319f2bd998a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:30.210181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689aae3e-fe37-4577-a246-319f2bd998a2 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:30.210839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689aae3e-fe37-4577-a246-319f2bd998a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:30.212341 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689aae3e-fe37-4577-a246-319f2bd998a2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:30.339060 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 29/61] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:04:30 2022] POST /identity/v3/users => generated 275 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:04:33.615325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bc1fd-faaa-4269-b539-f165c7f38471 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:33.616346 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bc1fd-faaa-4269-b539-f165c7f38471 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:33.616832 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bc1fd-faaa-4269-b539-f165c7f38471 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:33.618394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 33/62] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:33.624809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b453fe3-e5f1-4671-979e-ec92faed1fb2 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:33.625746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b453fe3-e5f1-4671-979e-ec92faed1fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:33.626246 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b453fe3-e5f1-4671-979e-ec92faed1fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:33.656233 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8b453fe3-e5f1-4671-979e-ec92faed1fb2 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:33.657465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8b453fe3-e5f1-4671-979e-ec92faed1fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:33.738357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44d8ba-bd57-471c-936e-38db89fdf8a1 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:33.738984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44d8ba-bd57-471c-936e-38db89fdf8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:33.739489 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44d8ba-bd57-471c-936e-38db89fdf8a1 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:33.741275 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 34/63] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:33.746833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 30/64] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:33.747933 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe9485-f378-48ad-b14f-71b2e1cd6470 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:33.748426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe9485-f378-48ad-b14f-71b2e1cd6470 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:33.748911 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe9485-f378-48ad-b14f-71b2e1cd6470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:33.768592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-cabe9485-f378-48ad-b14f-71b2e1cd6470 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:33.770125 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-cabe9485-f378-48ad-b14f-71b2e1cd6470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:33.842109 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 35/65] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:34.042127 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bce311-acb5-4448-b779-6318b6c06833 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:34.042417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bce311-acb5-4448-b779-6318b6c06833 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:34.043470 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bce311-acb5-4448-b779-6318b6c06833 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:34.045010 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 31/66] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:04:34.061835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bbc66d-9c52-478b-94ff-c1d13658bcc0 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:34.062131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bbc66d-9c52-478b-94ff-c1d13658bcc0 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:34.062420 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bbc66d-9c52-478b-94ff-c1d13658bcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:34.075023 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:34.092243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-23bbc66d-9c52-478b-94ff-c1d13658bcc0 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:34.093576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-23bbc66d-9c52-478b-94ff-c1d13658bcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:34.134511 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bce311-acb5-4448-b779-6318b6c06833 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:34.199565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:34.213469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 36/67] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:34.250076 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bbc66d-9c52-478b-94ff-c1d13658bcc0 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:34.302428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bce311-acb5-4448-b779-6318b6c06833 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:34.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257a9d6-866c-4ad9-ba69-1139d1fa3ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:34.358406 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257a9d6-866c-4ad9-ba69-1139d1fa3ceb None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:34.358836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257a9d6-866c-4ad9-ba69-1139d1fa3ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:34.359406 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257a9d6-866c-4ad9-ba69-1139d1fa3ceb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:34.372871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a257a9d6-866c-4ad9-ba69-1139d1fa3ceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:34.381677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-a257a9d6-866c-4ad9-ba69-1139d1fa3ceb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:04:34.381677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:34.381677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:34.381677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:34.381677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:34.381677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:34.381677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:34.381677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:34.381677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:34.381677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:34.381677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:34.381677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:34.381677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:34.381677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:34.381677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:34.381677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:34.383230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:34.383230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:34.383230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:34.383230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:34.383230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:34.383230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:34.383230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:34.383230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:34.383230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:34.383230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:34.383230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:34.383230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:34.383230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:34.383230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:34.383230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:34.383230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:34.384544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:34.384544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:34.384544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:34.384544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:34.384544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:34.384544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:34.384544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:34.384544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:34.384544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:34.384544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:34.384544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:04:34.384544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:04:34.388080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 32/68] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:04:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:04:34.390768 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef13534-27b5-49c0-bbda-5ac7d727a05d None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:34.391490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef13534-27b5-49c0-bbda-5ac7d727a05d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:34.392146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef13534-27b5-49c0-bbda-5ac7d727a05d None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:34.394625 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 33/69] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:34.396582 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef13534-27b5-49c0-bbda-5ac7d727a05d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:34.399895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bbc66d-9c52-478b-94ff-c1d13658bcc0 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:34.439273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:34.445073 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61784c-d47b-4465-9670-a52133ed56a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:34.445879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61784c-d47b-4465-9670-a52133ed56a9 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:34.446247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61784c-d47b-4465-9670-a52133ed56a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:34.446582 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61784c-d47b-4465-9670-a52133ed56a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:34.457634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61784c-d47b-4465-9670-a52133ed56a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:34.463005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-bd61784c-d47b-4465-9670-a52133ed56a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:04:34.463005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:34.463005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:34.463005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:34.463005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:34.463005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:34.463005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:34.463005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:34.463005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:34.463005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:34.463005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:34.463005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:34.463005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:34.463005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:34.463005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:34.463005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:34.463890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:34.463890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:34.463890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:34.463890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:34.463890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:34.463890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:34.463890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:34.463890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:34.463890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:34.463890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:34.463890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:34.463890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:34.463890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:34.463890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:34.463890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:34.463890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:34.465176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:34.465176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:34.465176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:34.465176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:34.465176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:34.465176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:34.465176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:34.465176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:34.465176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:34.465176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:34.465176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:04:34.465176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:04:34.465176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 37/70] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:34.466380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7ff41-93c8-48be-a6a8-22cd2ad51b66 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:34.466889 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7ff41-93c8-48be-a6a8-22cd2ad51b66 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:34.467318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7ff41-93c8-48be-a6a8-22cd2ad51b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:34.488196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-86f7ff41-93c8-48be-a6a8-22cd2ad51b66 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:34.489812 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-86f7ff41-93c8-48be-a6a8-22cd2ad51b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:34.515149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:34.523547 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef13534-27b5-49c0-bbda-5ac7d727a05d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:34.563998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35093810-57af-4e06-955b-7a8000570a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:34.564550 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35093810-57af-4e06-955b-7a8000570a6f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:34.564838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35093810-57af-4e06-955b-7a8000570a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:34.565121 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35093810-57af-4e06-955b-7a8000570a6f None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:34.565796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35093810-57af-4e06-955b-7a8000570a6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:34.566106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35093810-57af-4e06-955b-7a8000570a6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:34.567567 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35093810-57af-4e06-955b-7a8000570a6f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:34.569814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 38/71] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:34.576328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7ff41-93c8-48be-a6a8-22cd2ad51b66 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:34.577265 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 34/72] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:34.578292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35093810-57af-4e06-955b-7a8000570a6f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:34.688591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35093810-57af-4e06-955b-7a8000570a6f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:34.701338 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7ff41-93c8-48be-a6a8-22cd2ad51b66 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:34.727421 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e38c9-bc49-4e36-960c-51739fb7cb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:34.727775 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:34.729134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e38c9-bc49-4e36-960c-51739fb7cb0e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:34.729423 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e38c9-bc49-4e36-960c-51739fb7cb0e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:34.729705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e38c9-bc49-4e36-960c-51739fb7cb0e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:34.739216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac93297-3fa6-4ce6-9c07-cf88de391a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:34.739845 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac93297-3fa6-4ce6-9c07-cf88de391a3c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:34.740131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac93297-3fa6-4ce6-9c07-cf88de391a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:34.740410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac93297-3fa6-4ce6-9c07-cf88de391a3c None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:34.740623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e38c9-bc49-4e36-960c-51739fb7cb0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:34.741142 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac93297-3fa6-4ce6-9c07-cf88de391a3c None admin] RBAC: Adding query filter params (name=member) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:34.741451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac93297-3fa6-4ce6-9c07-cf88de391a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:34.742840 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac93297-3fa6-4ce6-9c07-cf88de391a3c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:34.747038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-243e38c9-bc49-4e36-960c-51739fb7cb0e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:04:34.747038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:34.747038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:34.747038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:34.747038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:34.747038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:34.747038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:34.747038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:34.747038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:34.747038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:34.747038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:34.747038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:34.747038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:34.747038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:34.747038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:34.747038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:34.748039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:34.748039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:34.748039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:34.748039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:34.748039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:34.748039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:34.748039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:34.748039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:34.748039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:34.748039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:34.748039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:34.748039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:34.748039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:34.748039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:34.748039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:34.748039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:34.748845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:34.748845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:34.748845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:34.748845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:34.748845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:34.748845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:34.748845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:34.748845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:34.748845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:34.748845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:34.748845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:04:34.748845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:04:34.748845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 35/73] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:04:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:34.749477 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e38c9-bc49-4e36-960c-51739fb7cb0e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:34.756129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 39/74] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:04:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:34.760564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f7620-6d52-41fc-bfec-a559edad00dd None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:34.761999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f7620-6d52-41fc-bfec-a559edad00dd None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:34.762552 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f7620-6d52-41fc-bfec-a559edad00dd None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:34.764456 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 40/75] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:34.767946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f7620-6d52-41fc-bfec-a559edad00dd None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:34.776562 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:34.865148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e38c9-bc49-4e36-960c-51739fb7cb0e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:34.885502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f7620-6d52-41fc-bfec-a559edad00dd None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:34.899556 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1c49f-f06b-4e52-aef3-524d76ff6c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:34.900233 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1c49f-f06b-4e52-aef3-524d76ff6c58 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:34.900523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1c49f-f06b-4e52-aef3-524d76ff6c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:34.900803 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1c49f-f06b-4e52-aef3-524d76ff6c58 None admin] PATH_INFO: `/v3/users/b83cd9cf7ce84815942f0d89b245aefe` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:34.901950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1c49f-f06b-4e52-aef3-524d76ff6c58 None admin] RBAC: Authorizing `identity:get_user(user_id=b83cd9cf7ce84815942f0d89b245aefe)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:34.904028 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1c49f-f06b-4e52-aef3-524d76ff6c58 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:34.905621 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 36/76] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:34 2022] GET /identity/v3/users/b83cd9cf7ce84815942f0d89b245aefe => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:34.906395 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1c49f-f06b-4e52-aef3-524d76ff6c58 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:34.921431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf715c-3032-40ac-83fe-2141c74acb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:34.922056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf715c-3032-40ac-83fe-2141c74acb3e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:34.922339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf715c-3032-40ac-83fe-2141c74acb3e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:34.922617 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf715c-3032-40ac-83fe-2141c74acb3e None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:34.923368 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf715c-3032-40ac-83fe-2141c74acb3e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:34.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf715c-3032-40ac-83fe-2141c74acb3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:34.925034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf715c-3032-40ac-83fe-2141c74acb3e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:34.933130 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 41/77] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:04:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:34.934860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab14005-22e1-4b5c-abad-12fe97743f08 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:34.935147 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab14005-22e1-4b5c-abad-12fe97743f08 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:34.935456 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab14005-22e1-4b5c-abad-12fe97743f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:34.953273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-2ab14005-22e1-4b5c-abad-12fe97743f08 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:34.954475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2ab14005-22e1-4b5c-abad-12fe97743f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:34.995135 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:34.998882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:35.014195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1c49f-f06b-4e52-aef3-524d76ff6c58 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:35.019751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 42/78] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:35.020588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab14005-22e1-4b5c-abad-12fe97743f08 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:35.048831 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada6c9f-aa2a-4e1f-bb9f-335353157982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:35.049489 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada6c9f-aa2a-4e1f-bb9f-335353157982 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:35.049779 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada6c9f-aa2a-4e1f-bb9f-335353157982 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:35.050063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada6c9f-aa2a-4e1f-bb9f-335353157982 None admin] PATH_INFO: `/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:35.067720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada6c9f-aa2a-4e1f-bb9f-335353157982 None admin] RBAC: Authorizing `identity:get_user(user_id=3cecb6cf43464e65ba7e69e9eb50ccbb)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:35.069217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada6c9f-aa2a-4e1f-bb9f-335353157982 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:35.070922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 37/79] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:34 2022] GET /identity/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:35.071733 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada6c9f-aa2a-4e1f-bb9f-335353157982 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:35.144911 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab14005-22e1-4b5c-abad-12fe97743f08 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:35.181168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df25ea-b071-4e3b-bae5-4735f1f87f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:35.181806 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df25ea-b071-4e3b-bae5-4735f1f87f21 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:35.182091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df25ea-b071-4e3b-bae5-4735f1f87f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:35.182368 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df25ea-b071-4e3b-bae5-4735f1f87f21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:35.186216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada6c9f-aa2a-4e1f-bb9f-335353157982 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:35.193881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03df25ea-b071-4e3b-bae5-4735f1f87f21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:35.198877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-03df25ea-b071-4e3b-bae5-4735f1f87f21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:04:35.198877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:35.198877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:35.198877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:35.198877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:35.198877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:35.198877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:35.198877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:35.198877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:35.198877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:35.198877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:35.198877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:35.198877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:35.198877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:35.198877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:35.198877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:35.200352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:35.200352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:35.200352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:35.200352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:35.200352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:35.200352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:35.200352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:35.200352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:35.200352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:35.200352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:35.200352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:35.200352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:35.200352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:35.200352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:35.200352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:35.200352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:35.201117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:35.201117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:35.201117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:35.201117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:35.201117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:35.201117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:35.201117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:35.201117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:35.201117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:35.201117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:35.201117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:04:35.201117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:04:35.201117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 43/80] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:35.201739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df25ea-b071-4e3b-bae5-4735f1f87f21 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:35.230864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a11dd0-8ea7-462e-b9c3-0ee269e73db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:35.231808 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a11dd0-8ea7-462e-b9c3-0ee269e73db2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:35.232095 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a11dd0-8ea7-462e-b9c3-0ee269e73db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:35.232372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a11dd0-8ea7-462e-b9c3-0ee269e73db2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:35.240679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:35.249066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:35.303493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a11dd0-8ea7-462e-b9c3-0ee269e73db2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:35.304984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a11dd0-8ea7-462e-b9c3-0ee269e73db2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:35.312378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df25ea-b071-4e3b-bae5-4735f1f87f21 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:35.346036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2d176-81ff-48d8-a8ee-5a675f500f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:35.346663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2d176-81ff-48d8-a8ee-5a675f500f8d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:35.346943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2d176-81ff-48d8-a8ee-5a675f500f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:35.347236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2d176-81ff-48d8-a8ee-5a675f500f8d None admin] PATH_INFO: `/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:35.348374 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d2d176-81ff-48d8-a8ee-5a675f500f8d None admin] RBAC: Authorizing `identity:get_user(user_id=3cecb6cf43464e65ba7e69e9eb50ccbb)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:35.349817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d2d176-81ff-48d8-a8ee-5a675f500f8d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:35.351501 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 44/81] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:35 2022] GET /identity/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:35.352200 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2d176-81ff-48d8-a8ee-5a675f500f8d None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:35.408810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 38/82] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:04:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:04:35.409600 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a11dd0-8ea7-462e-b9c3-0ee269e73db2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:35.455634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:35.460872 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2d176-81ff-48d8-a8ee-5a675f500f8d None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:35.463161 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:35.493654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54394e6-0301-4376-9db1-7cb09e11a753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:35.494267 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54394e6-0301-4376-9db1-7cb09e11a753 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:35.494545 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54394e6-0301-4376-9db1-7cb09e11a753 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:35.494817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54394e6-0301-4376-9db1-7cb09e11a753 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:35.509476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a11dd0-8ea7-462e-b9c3-0ee269e73db2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:35.542252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e2f3d-e49c-4b05-8d82-892a6277ce4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:35.543418 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e2f3d-e49c-4b05-8d82-892a6277ce4c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:35.543708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e2f3d-e49c-4b05-8d82-892a6277ce4c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:35.543972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e2f3d-e49c-4b05-8d82-892a6277ce4c None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:35.544634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e2f3d-e49c-4b05-8d82-892a6277ce4c None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:35.544934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e2f3d-e49c-4b05-8d82-892a6277ce4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:35.546338 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e2f3d-e49c-4b05-8d82-892a6277ce4c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:35.554224 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 39/83] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:04:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:35.555033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e2f3d-e49c-4b05-8d82-892a6277ce4c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:35.566940 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54394e6-0301-4376-9db1-7cb09e11a753 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:35.568441 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54394e6-0301-4376-9db1-7cb09e11a753 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:35.653183 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e2f3d-e49c-4b05-8d82-892a6277ce4c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:35.660162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:35.666641 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 45/84] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:04:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:04:35.666641 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54394e6-0301-4376-9db1-7cb09e11a753 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:35.666641 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:35.692965 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b9676-54a9-426c-8f29-3d0ef3ecdbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:35.693605 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b9676-54a9-426c-8f29-3d0ef3ecdbbd None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:35.693891 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b9676-54a9-426c-8f29-3d0ef3ecdbbd None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:35.694168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b9676-54a9-426c-8f29-3d0ef3ecdbbd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:35.695320 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b9676-54a9-426c-8f29-3d0ef3ecdbbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:35.696773 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b9676-54a9-426c-8f29-3d0ef3ecdbbd None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:35.698437 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 40/85] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:04:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:35.699280 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b9676-54a9-426c-8f29-3d0ef3ecdbbd None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:35.765283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54394e6-0301-4376-9db1-7cb09e11a753 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:35.800429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b9676-54a9-426c-8f29-3d0ef3ecdbbd None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:35.802212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76e44b-8560-4e6d-a501-c65d8a7ce7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:35.802832 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76e44b-8560-4e6d-a501-c65d8a7ce7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:35.803111 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76e44b-8560-4e6d-a501-c65d8a7ce7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:35.803435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76e44b-8560-4e6d-a501-c65d8a7ce7c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:35.834398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00614f36-b0e3-4bdb-b5b2-5fd98ca5062f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:35.835058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00614f36-b0e3-4bdb-b5b2-5fd98ca5062f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:35.835377 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00614f36-b0e3-4bdb-b5b2-5fd98ca5062f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:35.835655 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00614f36-b0e3-4bdb-b5b2-5fd98ca5062f None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:35.836799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00614f36-b0e3-4bdb-b5b2-5fd98ca5062f None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:35.838341 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00614f36-b0e3-4bdb-b5b2-5fd98ca5062f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:35.840546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 41/86] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:35 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:35.841324 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00614f36-b0e3-4bdb-b5b2-5fd98ca5062f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:35.872237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76e44b-8560-4e6d-a501-c65d8a7ce7c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:35.873736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76e44b-8560-4e6d-a501-c65d8a7ce7c7 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:35.876819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:35.888135 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:35.946090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00614f36-b0e3-4bdb-b5b2-5fd98ca5062f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:35.972237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 46/87] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:04:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:04:35.973186 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76e44b-8560-4e6d-a501-c65d8a7ce7c7 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:35.982628 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f89052-fe09-47b5-ad9e-9f96e3802ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:35.983306 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f89052-fe09-47b5-ad9e-9f96e3802ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:35.983595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f89052-fe09-47b5-ad9e-9f96e3802ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:35.983880 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f89052-fe09-47b5-ad9e-9f96e3802ac7 None admin] PATH_INFO: `/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:35.985049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f89052-fe09-47b5-ad9e-9f96e3802ac7 None admin] RBAC: Authorizing `identity:get_user(user_id=3cecb6cf43464e65ba7e69e9eb50ccbb)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:35.986516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f89052-fe09-47b5-ad9e-9f96e3802ac7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:35.988138 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 42/88] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:35 2022] GET /identity/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:35.988886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f89052-fe09-47b5-ad9e-9f96e3802ac7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:36.080778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76e44b-8560-4e6d-a501-c65d8a7ce7c7 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:36.083093 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:36.088783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f89052-fe09-47b5-ad9e-9f96e3802ac7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:36.099702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:36.115476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c9110-327b-403e-aebd-6d7b189527be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:36.115662 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c9110-327b-403e-aebd-6d7b189527be None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:36.115889 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c9110-327b-403e-aebd-6d7b189527be None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:36.116163 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c9110-327b-403e-aebd-6d7b189527be None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:36.117308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c9110-327b-403e-aebd-6d7b189527be None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:36.118760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c9110-327b-403e-aebd-6d7b189527be None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:36.120430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 47/89] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:35 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:36.123077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a88268-863c-4f28-991c-b5a5d9fc3e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:36.123741 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a88268-863c-4f28-991c-b5a5d9fc3e8f None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:36.124029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a88268-863c-4f28-991c-b5a5d9fc3e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:36.124310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a88268-863c-4f28-991c-b5a5d9fc3e8f None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:36.124966 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a88268-863c-4f28-991c-b5a5d9fc3e8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:36.126375 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c9110-327b-403e-aebd-6d7b189527be None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:36.126690 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a88268-863c-4f28-991c-b5a5d9fc3e8f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:36.224532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c9110-327b-403e-aebd-6d7b189527be None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:36.229407 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 43/90] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:04:35 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:04:36.230237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a88268-863c-4f28-991c-b5a5d9fc3e8f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:36.263385 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6df1d-a6c7-426e-8693-43d10aa68e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:36.264015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6df1d-a6c7-426e-8693-43d10aa68e3e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:36.264327 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6df1d-a6c7-426e-8693-43d10aa68e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:36.264604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6df1d-a6c7-426e-8693-43d10aa68e3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:36.265403 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6df1d-a6c7-426e-8693-43d10aa68e3e None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=ff6a8ae52b074b64be28a144288b4780, user.id=b83cd9cf7ce84815942f0d89b245aefe) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:36.265667 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6df1d-a6c7-426e-8693-43d10aa68e3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:36.267156 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6df1d-a6c7-426e-8693-43d10aa68e3e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:36.273594 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 48/91] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:04:36 2022] GET /identity/v3/role_assignments?user.id=b83cd9cf7ce84815942f0d89b245aefe&scope.project.id=ff6a8ae52b074b64be28a144288b4780&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:36.276519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6df1d-a6c7-426e-8693-43d10aa68e3e None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:36.320510 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:36.352904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a88268-863c-4f28-991c-b5a5d9fc3e8f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:36.393903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f467f3-1d5f-4bc0-8274-45491f0c4868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:36.394570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f467f3-1d5f-4bc0-8274-45491f0c4868 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:36.394855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f467f3-1d5f-4bc0-8274-45491f0c4868 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:36.395134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f467f3-1d5f-4bc0-8274-45491f0c4868 None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:36.396485 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f467f3-1d5f-4bc0-8274-45491f0c4868 None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:36.398048 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f467f3-1d5f-4bc0-8274-45491f0c4868 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:36.400270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 44/92] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:36 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:36.401165 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f467f3-1d5f-4bc0-8274-45491f0c4868 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:36.407517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6df1d-a6c7-426e-8693-43d10aa68e3e None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:36.444721 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:36.455042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89b0b7-d803-4c53-83d2-aac71730dd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:36.456920 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89b0b7-d803-4c53-83d2-aac71730dd34 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:36.456920 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89b0b7-d803-4c53-83d2-aac71730dd34 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:36.456920 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89b0b7-d803-4c53-83d2-aac71730dd34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:36.537327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f467f3-1d5f-4bc0-8274-45491f0c4868 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:36.539780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89b0b7-d803-4c53-83d2-aac71730dd34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:36.541497 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89b0b7-d803-4c53-83d2-aac71730dd34 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:36.589359 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6afa81-d57f-4ceb-b41e-9f56fd9a8309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:36.590101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6afa81-d57f-4ceb-b41e-9f56fd9a8309 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:36.590700 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6afa81-d57f-4ceb-b41e-9f56fd9a8309 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:36.590991 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6afa81-d57f-4ceb-b41e-9f56fd9a8309 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:36.591861 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6afa81-d57f-4ceb-b41e-9f56fd9a8309 None admin] RBAC: Adding query filter params (role.id=486614b0ab4b454b92e9ef0bc7ec61c1, scope.project.id=ff6a8ae52b074b64be28a144288b4780, user.id=3cecb6cf43464e65ba7e69e9eb50ccbb) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:36.592346 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:36.593412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6afa81-d57f-4ceb-b41e-9f56fd9a8309 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:36.594936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6afa81-d57f-4ceb-b41e-9f56fd9a8309 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:36.602103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 45/93] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:04:36 2022] GET /identity/v3/role_assignments?user.id=3cecb6cf43464e65ba7e69e9eb50ccbb&scope.project.id=ff6a8ae52b074b64be28a144288b4780&role.id=486614b0ab4b454b92e9ef0bc7ec61c1 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:36.605973 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6afa81-d57f-4ceb-b41e-9f56fd9a8309 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:36.718147 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 49/94] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:04:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:04:36.729490 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89b0b7-d803-4c53-83d2-aac71730dd34 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:36.741316 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:36.745507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6afa81-d57f-4ceb-b41e-9f56fd9a8309 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:36.835137 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedaf35-4ba0-4794-b033-ae89c3d21539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:36.835716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedaf35-4ba0-4794-b033-ae89c3d21539 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:36.836001 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedaf35-4ba0-4794-b033-ae89c3d21539 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:36.836274 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedaf35-4ba0-4794-b033-ae89c3d21539 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:36.837026 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efedaf35-4ba0-4794-b033-ae89c3d21539 None admin] RBAC: Adding query filter params (role.id=b7eccd60e28d4b39b54879dad8230e0d, scope.project.id=ff6a8ae52b074b64be28a144288b4780, user.id=3cecb6cf43464e65ba7e69e9eb50ccbb) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:36.837334 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efedaf35-4ba0-4794-b033-ae89c3d21539 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:36.838796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efedaf35-4ba0-4794-b033-ae89c3d21539 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:36.850976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 46/95] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:04:36 2022] GET /identity/v3/role_assignments?user.id=3cecb6cf43464e65ba7e69e9eb50ccbb&scope.project.id=ff6a8ae52b074b64be28a144288b4780&role.id=b7eccd60e28d4b39b54879dad8230e0d => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:36.926721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89b0b7-d803-4c53-83d2-aac71730dd34 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:36.978892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c97835-d9f2-4ea7-a6d4-c31bcdc5aca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:36.979588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c97835-d9f2-4ea7-a6d4-c31bcdc5aca3 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:36.979871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c97835-d9f2-4ea7-a6d4-c31bcdc5aca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:36.980726 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c97835-d9f2-4ea7-a6d4-c31bcdc5aca3 None admin] PATH_INFO: `/v3/projects/b7499478eb9e4bd9bff19696f61ceff5` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:36.982277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c97835-d9f2-4ea7-a6d4-c31bcdc5aca3 None admin] RBAC: Authorizing `identity:get_project(project_id=b7499478eb9e4bd9bff19696f61ceff5)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:36.983877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c97835-d9f2-4ea7-a6d4-c31bcdc5aca3 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:36.985881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 50/96] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:36 2022] GET /identity/v3/projects/b7499478eb9e4bd9bff19696f61ceff5 => generated 312 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:36.992034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedaf35-4ba0-4794-b033-ae89c3d21539 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:37.141426 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedaf35-4ba0-4794-b033-ae89c3d21539 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:37.196918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366ee8d-ba8b-4809-bf7b-b46f756b45bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:37.197583 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366ee8d-ba8b-4809-bf7b-b46f756b45bc None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:37.197867 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366ee8d-ba8b-4809-bf7b-b46f756b45bc None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:37.198144 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366ee8d-ba8b-4809-bf7b-b46f756b45bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:37.198929 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f366ee8d-ba8b-4809-bf7b-b46f756b45bc None admin] RBAC: Adding query filter params (role.id=486614b0ab4b454b92e9ef0bc7ec61c1, scope.project.id=b7499478eb9e4bd9bff19696f61ceff5, user.id=3cecb6cf43464e65ba7e69e9eb50ccbb) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:37.199270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f366ee8d-ba8b-4809-bf7b-b46f756b45bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:37.200761 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f366ee8d-ba8b-4809-bf7b-b46f756b45bc None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:37.211663 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 47/97] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:04:36 2022] GET /identity/v3/role_assignments?user.id=3cecb6cf43464e65ba7e69e9eb50ccbb&scope.project.id=b7499478eb9e4bd9bff19696f61ceff5&role.id=486614b0ab4b454b92e9ef0bc7ec61c1 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:38.792020 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9603b4-41f6-4f5f-8d64-d9b3333f7241 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:38.792552 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9603b4-41f6-4f5f-8d64-d9b3333f7241 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:38.792730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9603b4-41f6-4f5f-8d64-d9b3333f7241 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:38.793944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 51/98] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:38.799693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a634d-a0dd-46b3-94a8-848678498277 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:38.800421 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a634d-a0dd-46b3-94a8-848678498277 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:38.800421 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a634d-a0dd-46b3-94a8-848678498277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:38.827250 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-086a634d-a0dd-46b3-94a8-848678498277 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:38.828463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-086a634d-a0dd-46b3-94a8-848678498277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:38.859311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:38.909732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 48/99] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:39.531493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebcd8d-0b6e-4d79-9da9-2eac7ba20823 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:39.531899 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebcd8d-0b6e-4d79-9da9-2eac7ba20823 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:39.532076 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebcd8d-0b6e-4d79-9da9-2eac7ba20823 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:39.533868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 52/100] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:39.539334 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821e36d-45b0-4db1-89e8-2d30dea16b13 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:39.539623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821e36d-45b0-4db1-89e8-2d30dea16b13 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:39.539902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821e36d-45b0-4db1-89e8-2d30dea16b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:39.550368 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebcd8d-0b6e-4d79-9da9-2eac7ba20823 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:39.555280 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:39.571349 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d821e36d-45b0-4db1-89e8-2d30dea16b13 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:39.572555 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d821e36d-45b0-4db1-89e8-2d30dea16b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:39.659729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 49/101] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:39.671879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298802a-976c-4a70-8d84-dbe33b4dff8b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:39.672545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298802a-976c-4a70-8d84-dbe33b4dff8b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:39.673026 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298802a-976c-4a70-8d84-dbe33b4dff8b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:39.674574 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 50/102] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:04:39.679996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95155b-c154-4675-8127-0c65c949a55f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:39.680556 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95155b-c154-4675-8127-0c65c949a55f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:39.681033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95155b-c154-4675-8127-0c65c949a55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:39.700199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-cc95155b-c154-4675-8127-0c65c949a55f None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:39.701801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-cc95155b-c154-4675-8127-0c65c949a55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:39.775782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 51/103] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 19 11:04:39.818088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebcd8d-0b6e-4d79-9da9-2eac7ba20823 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:39.857268 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aea97e-3fde-4d6d-a45f-f2af49e09f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:39.857949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aea97e-3fde-4d6d-a45f-f2af49e09f43 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:39.858231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aea97e-3fde-4d6d-a45f-f2af49e09f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:39.858504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aea97e-3fde-4d6d-a45f-f2af49e09f43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:39.869456 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aea97e-3fde-4d6d-a45f-f2af49e09f43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:39.875437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-c2aea97e-3fde-4d6d-a45f-f2af49e09f43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:04:39.875437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:39.875437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:39.875437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:39.875437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:39.875437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:39.875437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:39.875437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:39.875437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:39.875437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:39.875437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:39.875437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:39.875437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:39.875437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:39.875437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:39.875437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:39.876399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:39.876399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:39.876399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:39.876399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:39.876399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:39.876399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:39.876399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:39.876399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:39.876399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:39.876399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:39.876399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:39.876399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:39.876399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:39.876399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:39.876399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:39.876399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:39.877166 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:39.877166 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:39.877166 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:39.877166 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:39.877166 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:39.877166 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:39.877166 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:39.877166 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:39.877166 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:39.877166 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:39.877166 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:04:39.877166 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:04:39.877166 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 53/104] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:04:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:39.883241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95155b-c154-4675-8127-0c65c949a55f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:39.903924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:39.986622 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95155b-c154-4675-8127-0c65c949a55f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:40.026424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de40bf7-6605-4e5c-9c8e-cf526b45eb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:40.027630 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de40bf7-6605-4e5c-9c8e-cf526b45eb69 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:40.027907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de40bf7-6605-4e5c-9c8e-cf526b45eb69 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:40.028183 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de40bf7-6605-4e5c-9c8e-cf526b45eb69 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:40.028856 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de40bf7-6605-4e5c-9c8e-cf526b45eb69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:40.029158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de40bf7-6605-4e5c-9c8e-cf526b45eb69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:40.030530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de40bf7-6605-4e5c-9c8e-cf526b45eb69 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:40.038105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfbc45-2096-4163-ab32-c30629ec8dc3 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:40.038379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfbc45-2096-4163-ab32-c30629ec8dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:40.038657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfbc45-2096-4163-ab32-c30629ec8dc3 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:40.040170 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 54/105] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:04:40.043953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 52/106] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:40.046450 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2733c-d7e5-4962-920e-9bf2b4b552a7 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:40.046736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2733c-d7e5-4962-920e-9bf2b4b552a7 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:40.047006 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2733c-d7e5-4962-920e-9bf2b4b552a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:40.049644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de40bf7-6605-4e5c-9c8e-cf526b45eb69 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:40.084360 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-c7a2733c-d7e5-4962-920e-9bf2b4b552a7 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:40.085875 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c7a2733c-d7e5-4962-920e-9bf2b4b552a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:40.165009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:40.175238 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 55/107] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:40.176722 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2733c-d7e5-4962-920e-9bf2b4b552a7 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:40.215197 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:40.231678 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de40bf7-6605-4e5c-9c8e-cf526b45eb69 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:40.277303 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d98f80-80ec-4385-8994-3e5ddeb23296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:40.277957 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d98f80-80ec-4385-8994-3e5ddeb23296 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:40.278243 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d98f80-80ec-4385-8994-3e5ddeb23296 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:40.278523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d98f80-80ec-4385-8994-3e5ddeb23296 None admin] PATH_INFO: `/v3/users/b83cd9cf7ce84815942f0d89b245aefe` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:40.279846 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d98f80-80ec-4385-8994-3e5ddeb23296 None admin] RBAC: Authorizing `identity:get_user(user_id=b83cd9cf7ce84815942f0d89b245aefe)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:40.281305 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d98f80-80ec-4385-8994-3e5ddeb23296 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:40.283071 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 53/108] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:40 2022] GET /identity/v3/users/b83cd9cf7ce84815942f0d89b245aefe => generated 247 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:40.283711 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d98f80-80ec-4385-8994-3e5ddeb23296 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:40.334551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2733c-d7e5-4962-920e-9bf2b4b552a7 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:40.378512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225529a8-3432-4424-a78c-4a7e8e56bf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:40.379782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225529a8-3432-4424-a78c-4a7e8e56bf30 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:40.382135 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225529a8-3432-4424-a78c-4a7e8e56bf30 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:40.382135 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225529a8-3432-4424-a78c-4a7e8e56bf30 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:40.395239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225529a8-3432-4424-a78c-4a7e8e56bf30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:40.401415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-225529a8-3432-4424-a78c-4a7e8e56bf30 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:04:40.401415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:40.401415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:40.401415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:40.401415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:40.401415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:40.401415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:40.401415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:40.401415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:40.401415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:40.401415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:40.401415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:40.401415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:40.401415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:40.401415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:40.401415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:40.402358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:40.402358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:40.402358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:40.402358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:40.402358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:40.402358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:40.402358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:40.402358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:40.402358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:40.402358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:40.402358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:40.402358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:40.402358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:40.402358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:40.402358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:40.402358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:40.403139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:40.403139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:40.403139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:40.403139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:40.403139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:40.403139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:40.403139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:40.403139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:40.403139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:40.403139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:40.403139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:04:40.403139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:04:40.406388 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 56/109] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:40.407835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225529a8-3432-4424-a78c-4a7e8e56bf30 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:40.495119 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:40.534770 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d98f80-80ec-4385-8994-3e5ddeb23296 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:40.549468 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225529a8-3432-4424-a78c-4a7e8e56bf30 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:40.586213 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40b107-74d1-4137-ad38-6206aee23789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:40.586811 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a1cd2-d89b-4acc-9881-f7ab3efeac05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:40.587467 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40b107-74d1-4137-ad38-6206aee23789 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:40.587572 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a1cd2-d89b-4acc-9881-f7ab3efeac05 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:40.587767 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a1cd2-d89b-4acc-9881-f7ab3efeac05 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:40.588046 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a1cd2-d89b-4acc-9881-f7ab3efeac05 None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:40.588448 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40b107-74d1-4137-ad38-6206aee23789 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:40.588935 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40b107-74d1-4137-ad38-6206aee23789 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:40.589193 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a1cd2-d89b-4acc-9881-f7ab3efeac05 None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:40.590518 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40b107-74d1-4137-ad38-6206aee23789 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:40.590704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a1cd2-d89b-4acc-9881-f7ab3efeac05 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:40.592374 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40b107-74d1-4137-ad38-6206aee23789 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:40.592778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 57/110] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:40 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:40.593574 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a1cd2-d89b-4acc-9881-f7ab3efeac05 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:40.594724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 54/111] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:04:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:40.595752 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40b107-74d1-4137-ad38-6206aee23789 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:40.621384 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:40.701162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a1cd2-d89b-4acc-9881-f7ab3efeac05 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:40.711708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40b107-74d1-4137-ad38-6206aee23789 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:40.724068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:40.736221 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6da2f-20a6-4e1e-be78-4baca2c3912d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:40.736884 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6da2f-20a6-4e1e-be78-4baca2c3912d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:40.737168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6da2f-20a6-4e1e-be78-4baca2c3912d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:40.737449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6da2f-20a6-4e1e-be78-4baca2c3912d None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:40.738120 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b6da2f-20a6-4e1e-be78-4baca2c3912d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:40.738427 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b6da2f-20a6-4e1e-be78-4baca2c3912d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:40.739884 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b6da2f-20a6-4e1e-be78-4baca2c3912d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:40.748714 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 58/112] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:04:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:40.749501 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6da2f-20a6-4e1e-be78-4baca2c3912d None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:40.750191 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec6cc8-b1fb-4e67-98e6-18ce31655ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:40.750997 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec6cc8-b1fb-4e67-98e6-18ce31655ece None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:40.751301 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec6cc8-b1fb-4e67-98e6-18ce31655ece None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:40.751574 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec6cc8-b1fb-4e67-98e6-18ce31655ece None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:40.762734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec6cc8-b1fb-4e67-98e6-18ce31655ece None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:40.768483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-f4ec6cc8-b1fb-4e67-98e6-18ce31655ece None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:04:40.768483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:40.768483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:40.768483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:40.768483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:40.768483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:40.768483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:40.768483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:40.768483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:40.768483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:40.768483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:40.768483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:40.768483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:40.768483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:40.768483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:40.768483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:40.769893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:40.769893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:40.769893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:40.769893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:40.769893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:40.769893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:40.769893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:40.769893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:40.769893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:40.769893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:40.769893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:40.769893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:40.769893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:40.769893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:40.769893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:40.769893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:40.776446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:40.776446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:40.776446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:40.776446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:40.776446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:40.776446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:40.776446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:40.776446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:40.776446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:40.776446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:40.776446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:04:40.776446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:04:40.776446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 55/113] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:40.776446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92344041-9b3f-4b45-99f8-76a34247f273 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:40.777888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92344041-9b3f-4b45-99f8-76a34247f273 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:40.777888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92344041-9b3f-4b45-99f8-76a34247f273 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:40.777888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 56/114] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:40.779446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92344041-9b3f-4b45-99f8-76a34247f273 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:40.870040 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:40.872088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6da2f-20a6-4e1e-be78-4baca2c3912d None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:40.901207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92344041-9b3f-4b45-99f8-76a34247f273 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:40.912230 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea2f17-8326-45b0-ad13-3cf889a3ecfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:40.912909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea2f17-8326-45b0-ad13-3cf889a3ecfd None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:40.913187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea2f17-8326-45b0-ad13-3cf889a3ecfd None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:40.913475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea2f17-8326-45b0-ad13-3cf889a3ecfd None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780/users/b83cd9cf7ce84815942f0d89b245aefe/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:40.915689 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea2f17-8326-45b0-ad13-3cf889a3ecfd None admin] RBAC: Authorizing `identity:create_grant(project_id=ff6a8ae52b074b64be28a144288b4780, user_id=b83cd9cf7ce84815942f0d89b245aefe, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:40.918277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea2f17-8326-45b0-ad13-3cf889a3ecfd None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:40.939815 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3644b3-4a51-49f4-9d8f-180afa1adc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:40.940533 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3644b3-4a51-49f4-9d8f-180afa1adc4d None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:40.940825 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3644b3-4a51-49f4-9d8f-180afa1adc4d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:40.941102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3644b3-4a51-49f4-9d8f-180afa1adc4d None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:40.941757 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3644b3-4a51-49f4-9d8f-180afa1adc4d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:40.943292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3644b3-4a51-49f4-9d8f-180afa1adc4d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:40.995530 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 59/115] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:04:40 2022] PUT /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780/users/b83cd9cf7ce84815942f0d89b245aefe/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:04:40.996375 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea2f17-8326-45b0-ad13-3cf889a3ecfd None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:41.048753 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:41.054693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 57/116] 10.222.0.78 () {60 vars in 1141 bytes} [Mon Sep 19 11:04:40 2022] POST /identity/v3/projects => generated 302 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:04:41.068745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3644b3-4a51-49f4-9d8f-180afa1adc4d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:41.134370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea2f17-8326-45b0-ad13-3cf889a3ecfd None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:41.189726 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4b47f-0c1b-4836-bb2b-39a51bf81846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:41.190992 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4b47f-0c1b-4836-bb2b-39a51bf81846 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:41.191529 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4b47f-0c1b-4836-bb2b-39a51bf81846 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:41.192086 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4b47f-0c1b-4836-bb2b-39a51bf81846 None admin] PATH_INFO: `/v3/users/97b48b02dddd451687de91f70bf306ee` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:41.226851 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4b47f-0c1b-4836-bb2b-39a51bf81846 None admin] RBAC: Authorizing `identity:get_user(user_id=97b48b02dddd451687de91f70bf306ee)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:41.228727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4b47f-0c1b-4836-bb2b-39a51bf81846 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:41.231595 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 60/117] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:40 2022] GET /identity/v3/users/97b48b02dddd451687de91f70bf306ee => generated 289 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:41.233209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749ddef-890c-4572-b61c-b76a954dab75 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:41.233723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749ddef-890c-4572-b61c-b76a954dab75 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:41.234220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749ddef-890c-4572-b61c-b76a954dab75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:41.247719 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3644b3-4a51-49f4-9d8f-180afa1adc4d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:41.257066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-5749ddef-890c-4572-b61c-b76a954dab75 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:41.258681 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5749ddef-890c-4572-b61c-b76a954dab75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:41.278532 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:41.292043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92600a-5c6f-439b-91e2-391d020d135a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:41.293508 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92600a-5c6f-439b-91e2-391d020d135a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:41.293803 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92600a-5c6f-439b-91e2-391d020d135a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:41.294069 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92600a-5c6f-439b-91e2-391d020d135a None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:41.294753 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92600a-5c6f-439b-91e2-391d020d135a None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:41.295063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92600a-5c6f-439b-91e2-391d020d135a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:41.296483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92600a-5c6f-439b-91e2-391d020d135a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:41.307531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 58/118] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:04:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:41.308851 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a23002-c0cd-41d9-b2fd-3f5a80137a1c None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:41.309142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a23002-c0cd-41d9-b2fd-3f5a80137a1c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:41.309431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a23002-c0cd-41d9-b2fd-3f5a80137a1c None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:41.311108 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 59/119] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:41.316519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a23002-c0cd-41d9-b2fd-3f5a80137a1c None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:41.350269 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 61/120] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:41.351539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749ddef-890c-4572-b61c-b76a954dab75 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:41.385593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:41.458053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a23002-c0cd-41d9-b2fd-3f5a80137a1c None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:41.468411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749ddef-890c-4572-b61c-b76a954dab75 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:41.506574 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030f529-4ef0-4662-96db-bb7e9c6d669e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:41.507279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030f529-4ef0-4662-96db-bb7e9c6d669e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:41.511720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030f529-4ef0-4662-96db-bb7e9c6d669e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:41.511720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030f529-4ef0-4662-96db-bb7e9c6d669e None admin] PATH_INFO: `/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:41.511720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030f529-4ef0-4662-96db-bb7e9c6d669e None admin] RBAC: Authorizing `identity:get_user(user_id=3cecb6cf43464e65ba7e69e9eb50ccbb)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:41.511720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030f529-4ef0-4662-96db-bb7e9c6d669e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:41.513152 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 62/121] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:41 2022] GET /identity/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:41.514292 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e22748-d276-4424-86fd-073194ee2a92 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:41.514513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e22748-d276-4424-86fd-073194ee2a92 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:41.514795 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e22748-d276-4424-86fd-073194ee2a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:41.535131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-b0e22748-d276-4424-86fd-073194ee2a92 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:41.536523 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b0e22748-d276-4424-86fd-073194ee2a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:41.537175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c80a8-9bd3-43aa-a65b-6238773f10ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:41.538085 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c80a8-9bd3-43aa-a65b-6238773f10ff None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:41.538585 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c80a8-9bd3-43aa-a65b-6238773f10ff None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:41.539077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c80a8-9bd3-43aa-a65b-6238773f10ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:41.610652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 63/122] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:41.611589 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e22748-d276-4424-86fd-073194ee2a92 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:41.649996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:41.659585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:41.670519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c80a8-9bd3-43aa-a65b-6238773f10ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:41.672081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c80a8-9bd3-43aa-a65b-6238773f10ff None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:41.723732 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e22748-d276-4424-86fd-073194ee2a92 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:41.763108 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca9188-4ec6-4740-8b8c-b1fd9d0a9117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:41.764180 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca9188-4ec6-4740-8b8c-b1fd9d0a9117 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:41.764724 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca9188-4ec6-4740-8b8c-b1fd9d0a9117 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:41.765201 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca9188-4ec6-4740-8b8c-b1fd9d0a9117 None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:41.766662 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca9188-4ec6-4740-8b8c-b1fd9d0a9117 None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:41.768463 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca9188-4ec6-4740-8b8c-b1fd9d0a9117 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:41.770428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 64/123] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:41 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:41.784195 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca9188-4ec6-4740-8b8c-b1fd9d0a9117 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:41.819731 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 60/124] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:04:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:04:41.828184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c80a8-9bd3-43aa-a65b-6238773f10ff None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:41.935830 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca9188-4ec6-4740-8b8c-b1fd9d0a9117 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:41.939550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:41.972764 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:41.973526 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dea28c-7002-4c8d-aced-6cffdcf20aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:41.974384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dea28c-7002-4c8d-aced-6cffdcf20aac None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:41.975271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dea28c-7002-4c8d-aced-6cffdcf20aac None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:41.975271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dea28c-7002-4c8d-aced-6cffdcf20aac None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780/users/3cecb6cf43464e65ba7e69e9eb50ccbb/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:41.977817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dea28c-7002-4c8d-aced-6cffdcf20aac None admin] RBAC: Authorizing `identity:create_grant(project_id=ff6a8ae52b074b64be28a144288b4780, user_id=3cecb6cf43464e65ba7e69e9eb50ccbb, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:41.979570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dea28c-7002-4c8d-aced-6cffdcf20aac None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:41.985954 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c80a8-9bd3-43aa-a65b-6238773f10ff None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:42.024210 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa9ca8-8b83-4ba3-a1d8-5838cf29cbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:42.024862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa9ca8-8b83-4ba3-a1d8-5838cf29cbdf None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:42.025149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa9ca8-8b83-4ba3-a1d8-5838cf29cbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:42.025427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa9ca8-8b83-4ba3-a1d8-5838cf29cbdf None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:42.026570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa9ca8-8b83-4ba3-a1d8-5838cf29cbdf None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:42.028138 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa9ca8-8b83-4ba3-a1d8-5838cf29cbdf None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:42.029779 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 61/125] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:41 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:42.030640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa9ca8-8b83-4ba3-a1d8-5838cf29cbdf None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:42.094132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 65/126] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:04:41 2022] PUT /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780/users/3cecb6cf43464e65ba7e69e9eb50ccbb/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:04:42.095414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dea28c-7002-4c8d-aced-6cffdcf20aac None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:42.245364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa9ca8-8b83-4ba3-a1d8-5838cf29cbdf None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:42.267064 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dea28c-7002-4c8d-aced-6cffdcf20aac None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:42.305904 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f43c0-244d-4aca-bc21-cfc27a6edfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:42.306692 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f43c0-244d-4aca-bc21-cfc27a6edfb2 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:42.306975 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f43c0-244d-4aca-bc21-cfc27a6edfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:42.307401 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f43c0-244d-4aca-bc21-cfc27a6edfb2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:42.323124 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f43c0-244d-4aca-bc21-cfc27a6edfb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:42.323428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bf380-0c9e-4508-90aa-19478cc2dd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:42.324051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bf380-0c9e-4508-90aa-19478cc2dd1d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:42.324342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bf380-0c9e-4508-90aa-19478cc2dd1d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:42.324625 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bf380-0c9e-4508-90aa-19478cc2dd1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:42.328854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-0f4f43c0-244d-4aca-bc21-cfc27a6edfb2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:04:42.328854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:42.328854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:42.328854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:42.328854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:42.328854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:42.328854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:42.328854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:42.328854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:42.328854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:42.328854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:42.328854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:42.328854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:42.328854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:42.328854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:42.328854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:42.329717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:42.329717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:42.329717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:42.329717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:42.329717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:42.329717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:42.329717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:42.329717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:42.329717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:42.329717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:42.329717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:42.329717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:42.329717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:42.329717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:42.329717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:42.329717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:42.330476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:42.330476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:42.330476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:42.330476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:42.330476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:42.330476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:42.330476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:42.330476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:42.330476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:42.330476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:42.330476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:04:42.330476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:04:42.331131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 66/127] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:42.331757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f43c0-244d-4aca-bc21-cfc27a6edfb2 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:42.336331 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814bf380-0c9e-4508-90aa-19478cc2dd1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:42.341932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-814bf380-0c9e-4508-90aa-19478cc2dd1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:04:42.341932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:42.341932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:42.341932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:42.341932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:42.341932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:42.341932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:42.341932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:42.341932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:42.341932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:42.341932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:42.341932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:42.341932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:42.341932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:42.341932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:42.341932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:42.344038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:42.344038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:42.344038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:42.344038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:42.344038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:42.344038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:42.344038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:42.344038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:42.344038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:42.344038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:42.344038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:42.344038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:42.344038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:42.344038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:42.344038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:42.344038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:42.346130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:42.346130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:42.346130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:42.346130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:42.346130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:42.346130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:42.346130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:42.346130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:42.346130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:42.346130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:42.346130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:04:42.346130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:04:42.346130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 62/128] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:04:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:42.346130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bf380-0c9e-4508-90aa-19478cc2dd1d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:42.354101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:42.376008 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:42.465633 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bf380-0c9e-4508-90aa-19478cc2dd1d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:42.469738 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f43c0-244d-4aca-bc21-cfc27a6edfb2 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:42.509232 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cdd01-881e-45f8-91b4-96d8f9f0accc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:42.509922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cdd01-881e-45f8-91b4-96d8f9f0accc None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:42.510203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cdd01-881e-45f8-91b4-96d8f9f0accc None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:42.510550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cdd01-881e-45f8-91b4-96d8f9f0accc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:42.511429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6cdd01-881e-45f8-91b4-96d8f9f0accc None admin] RBAC: Adding query filter params (role.id=05303606367c4086bb8e3c0c20d45b56, scope.project.id=ff6a8ae52b074b64be28a144288b4780, user.id=97b48b02dddd451687de91f70bf306ee) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:42.511777 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6cdd01-881e-45f8-91b4-96d8f9f0accc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:42.513232 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6cdd01-881e-45f8-91b4-96d8f9f0accc None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:42.519473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded5e54-e7c7-4b62-a223-7cb34f773955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:42.520115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded5e54-e7c7-4b62-a223-7cb34f773955 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:42.520383 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 67/129] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:04:42 2022] GET /identity/v3/role_assignments?user.id=97b48b02dddd451687de91f70bf306ee&scope.project.id=ff6a8ae52b074b64be28a144288b4780&role.id=05303606367c4086bb8e3c0c20d45b56 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:42.520485 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded5e54-e7c7-4b62-a223-7cb34f773955 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:42.520672 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded5e54-e7c7-4b62-a223-7cb34f773955 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:42.521207 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cdd01-881e-45f8-91b4-96d8f9f0accc None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:42.521374 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded5e54-e7c7-4b62-a223-7cb34f773955 None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:42.521662 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded5e54-e7c7-4b62-a223-7cb34f773955 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:42.523071 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded5e54-e7c7-4b62-a223-7cb34f773955 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:42.538418 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 63/130] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:04:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:42.545321 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded5e54-e7c7-4b62-a223-7cb34f773955 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:42.683575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:42.711996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded5e54-e7c7-4b62-a223-7cb34f773955 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:42.726570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:42.776030 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cdd01-881e-45f8-91b4-96d8f9f0accc None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:42.795215 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2e9e8-d451-4bce-b47b-f677d3849df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:42.795885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2e9e8-d451-4bce-b47b-f677d3849df6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:42.796176 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2e9e8-d451-4bce-b47b-f677d3849df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:42.796460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2e9e8-d451-4bce-b47b-f677d3849df6 None admin] PATH_INFO: `/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:42.799096 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2e9e8-d451-4bce-b47b-f677d3849df6 None admin] RBAC: Authorizing `identity:get_user(user_id=3cecb6cf43464e65ba7e69e9eb50ccbb)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:42.808916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2e9e8-d451-4bce-b47b-f677d3849df6 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:42.810838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 64/131] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:42 2022] GET /identity/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:42.816232 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2e9e8-d451-4bce-b47b-f677d3849df6 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:42.872123 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ffba4-9257-4056-89cd-b2382c796cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:42.872756 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ffba4-9257-4056-89cd-b2382c796cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:42.873043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ffba4-9257-4056-89cd-b2382c796cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:42.873324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ffba4-9257-4056-89cd-b2382c796cb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:42.874001 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ffba4-9257-4056-89cd-b2382c796cb2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:42.874309 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ffba4-9257-4056-89cd-b2382c796cb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:42.875770 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ffba4-9257-4056-89cd-b2382c796cb2 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:42.891208 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 68/132] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:04:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:42.899405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ffba4-9257-4056-89cd-b2382c796cb2 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:42.977738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2e9e8-d451-4bce-b47b-f677d3849df6 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:43.016982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186da815-ad78-4daa-85e0-bc74dd7fc44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:43.018008 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186da815-ad78-4daa-85e0-bc74dd7fc44c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:43.018514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186da815-ad78-4daa-85e0-bc74dd7fc44c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:43.019014 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186da815-ad78-4daa-85e0-bc74dd7fc44c None admin] PATH_INFO: `/v3/projects/b7499478eb9e4bd9bff19696f61ceff5` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:43.020716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186da815-ad78-4daa-85e0-bc74dd7fc44c None admin] RBAC: Authorizing `identity:get_project(project_id=b7499478eb9e4bd9bff19696f61ceff5)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:43.023044 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186da815-ad78-4daa-85e0-bc74dd7fc44c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:43.024955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 65/133] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:42 2022] GET /identity/v3/projects/b7499478eb9e4bd9bff19696f61ceff5 => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:43.032255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186da815-ad78-4daa-85e0-bc74dd7fc44c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:43.063564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:43.178793 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ffba4-9257-4056-89cd-b2382c796cb2 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:43.190030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186da815-ad78-4daa-85e0-bc74dd7fc44c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:43.248616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51eee0-3bac-4bef-84c0-21f55cd53841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:43.249314 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51eee0-3bac-4bef-84c0-21f55cd53841 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:43.249607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51eee0-3bac-4bef-84c0-21f55cd53841 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:43.249891 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51eee0-3bac-4bef-84c0-21f55cd53841 None admin] PATH_INFO: `/v3/projects/b7499478eb9e4bd9bff19696f61ceff5/users/3cecb6cf43464e65ba7e69e9eb50ccbb/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:43.255961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51eee0-3bac-4bef-84c0-21f55cd53841 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7499478eb9e4bd9bff19696f61ceff5, user_id=3cecb6cf43464e65ba7e69e9eb50ccbb, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:43.257444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51eee0-3bac-4bef-84c0-21f55cd53841 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:43.276001 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509d189-22dc-418b-a746-531af4bb9b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:43.276660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509d189-22dc-418b-a746-531af4bb9b88 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:43.276979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509d189-22dc-418b-a746-531af4bb9b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:43.277632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509d189-22dc-418b-a746-531af4bb9b88 None admin] PATH_INFO: `/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:43.280489 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509d189-22dc-418b-a746-531af4bb9b88 None admin] RBAC: Authorizing `identity:get_user(user_id=3cecb6cf43464e65ba7e69e9eb50ccbb)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:43.282041 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509d189-22dc-418b-a746-531af4bb9b88 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:43.283788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 69/134] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:42 2022] GET /identity/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb => generated 275 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:43.289604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509d189-22dc-418b-a746-531af4bb9b88 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:43.314149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 66/135] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:04:43 2022] PUT /identity/v3/projects/b7499478eb9e4bd9bff19696f61ceff5/users/3cecb6cf43464e65ba7e69e9eb50ccbb/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:04:43.315551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:43.446574 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509d189-22dc-418b-a746-531af4bb9b88 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:43.500009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb4d18-3e37-4f5a-a8b7-67718bd68bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:43.500696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb4d18-3e37-4f5a-a8b7-67718bd68bcc None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:43.500984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb4d18-3e37-4f5a-a8b7-67718bd68bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:43.501267 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb4d18-3e37-4f5a-a8b7-67718bd68bcc None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:43.502600 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb4d18-3e37-4f5a-a8b7-67718bd68bcc None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:43.504129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb4d18-3e37-4f5a-a8b7-67718bd68bcc None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:43.506406 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 70/136] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:43 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:43.512777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51eee0-3bac-4bef-84c0-21f55cd53841 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:43.626110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:43.656547 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51eee0-3bac-4bef-84c0-21f55cd53841 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:43.693411 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71eeee-e91e-476f-ab28-a879e4637643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:43.694094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71eeee-e91e-476f-ab28-a879e4637643 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:43.694385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71eeee-e91e-476f-ab28-a879e4637643 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:43.694669 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71eeee-e91e-476f-ab28-a879e4637643 None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780/users/3cecb6cf43464e65ba7e69e9eb50ccbb/roles/b7eccd60e28d4b39b54879dad8230e0d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:43.697079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71eeee-e91e-476f-ab28-a879e4637643 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff6a8ae52b074b64be28a144288b4780, user_id=3cecb6cf43464e65ba7e69e9eb50ccbb, role_id=b7eccd60e28d4b39b54879dad8230e0d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:43.698581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71eeee-e91e-476f-ab28-a879e4637643 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:43.768350 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 67/137] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:04:43 2022] PUT /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780/users/3cecb6cf43464e65ba7e69e9eb50ccbb/roles/b7eccd60e28d4b39b54879dad8230e0d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:04:43.845137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64394d-6704-46bc-a5bb-6561d11c7743 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:43.845452 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64394d-6704-46bc-a5bb-6561d11c7743 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:43.845704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64394d-6704-46bc-a5bb-6561d11c7743 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:43.847303 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 71/138] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:43.857554 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833b825-11bd-45b1-a6c2-480aed397e60 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:43.857864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833b825-11bd-45b1-a6c2-480aed397e60 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:43.858129 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833b825-11bd-45b1-a6c2-480aed397e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:43.887464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b833b825-11bd-45b1-a6c2-480aed397e60 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:43.888648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b833b825-11bd-45b1-a6c2-480aed397e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:43.963052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 68/139] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:44.077185 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5fed6-0c75-48c0-b104-8d33ef870143 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:44.077499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5fed6-0c75-48c0-b104-8d33ef870143 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:44.077760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5fed6-0c75-48c0-b104-8d33ef870143 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:44.079151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 72/140] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:44.085982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f278d-f04a-4c84-b672-b961a3c86122 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:44.086473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f278d-f04a-4c84-b672-b961a3c86122 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:44.086545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f278d-f04a-4c84-b672-b961a3c86122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:44.112332 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-251f278d-f04a-4c84-b672-b961a3c86122 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:44.113670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-251f278d-f04a-4c84-b672-b961a3c86122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:44.196244 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:44.205009 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 69/141] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:44.341039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5fed6-0c75-48c0-b104-8d33ef870143 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:44.440811 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:44.467566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5fed6-0c75-48c0-b104-8d33ef870143 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:44.523333 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83af87-f6ef-4f51-b1f2-a94f79e95520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:44.524328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83af87-f6ef-4f51-b1f2-a94f79e95520 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:44.524803 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83af87-f6ef-4f51-b1f2-a94f79e95520 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:44.525269 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83af87-f6ef-4f51-b1f2-a94f79e95520 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:44.541922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83af87-f6ef-4f51-b1f2-a94f79e95520 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:44.547192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-cf83af87-f6ef-4f51-b1f2-a94f79e95520 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:04:44.547192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:44.547192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:44.547192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:44.547192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:44.547192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:44.547192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:44.547192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:44.547192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:44.547192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:44.547192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:44.547192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:44.547192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:44.547192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:44.547192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:44.547192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:44.548084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:44.548084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:44.548084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:44.548084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:44.548084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:44.548084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:44.548084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:44.548084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:44.548084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:44.548084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:44.548084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:44.548084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:44.548084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:44.548084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:44.548084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:44.548084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:44.548878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:44.548878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:44.548878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:44.548878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:44.548878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:44.548878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:44.548878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:44.548878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:44.548878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:44.548878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:44.548878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:04:44.548878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:04:44.551092 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 73/142] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:04:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:44.560296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f278d-f04a-4c84-b672-b961a3c86122 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:44.714684 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f278d-f04a-4c84-b672-b961a3c86122 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:44.765047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdd196-017d-4b1e-9777-ebc596930f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:44.765675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdd196-017d-4b1e-9777-ebc596930f29 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:44.765968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdd196-017d-4b1e-9777-ebc596930f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:44.766244 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdd196-017d-4b1e-9777-ebc596930f29 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:44.766925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdd196-017d-4b1e-9777-ebc596930f29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:44.767283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdd196-017d-4b1e-9777-ebc596930f29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:44.768684 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdd196-017d-4b1e-9777-ebc596930f29 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:44.783297 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 70/143] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:44.789160 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83af87-f6ef-4f51-b1f2-a94f79e95520 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:44.800174 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdd196-017d-4b1e-9777-ebc596930f29 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:44.903682 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:44.953158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdd196-017d-4b1e-9777-ebc596930f29 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:44.992401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62495e3d-85a8-4b39-880a-d5b9ac51cc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:44.993049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62495e3d-85a8-4b39-880a-d5b9ac51cc65 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:44.993330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62495e3d-85a8-4b39-880a-d5b9ac51cc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:44.993603 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62495e3d-85a8-4b39-880a-d5b9ac51cc65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:44.994157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83af87-f6ef-4f51-b1f2-a94f79e95520 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:44.994745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62495e3d-85a8-4b39-880a-d5b9ac51cc65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:44.996880 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62495e3d-85a8-4b39-880a-d5b9ac51cc65 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:44.998467 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 71/144] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:04:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:45.010008 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62495e3d-85a8-4b39-880a-d5b9ac51cc65 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:45.072305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7820a4-7112-45b1-af9c-542483c4f873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:45.072882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7820a4-7112-45b1-af9c-542483c4f873 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:45.073171 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7820a4-7112-45b1-af9c-542483c4f873 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:45.073448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7820a4-7112-45b1-af9c-542483c4f873 None admin] PATH_INFO: `/v3/users/b83cd9cf7ce84815942f0d89b245aefe` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:45.078417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7820a4-7112-45b1-af9c-542483c4f873 None admin] RBAC: Authorizing `identity:get_user(user_id=b83cd9cf7ce84815942f0d89b245aefe)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:45.080598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7820a4-7112-45b1-af9c-542483c4f873 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:45.082393 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 74/145] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:44 2022] GET /identity/v3/users/b83cd9cf7ce84815942f0d89b245aefe => generated 247 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:45.089261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7820a4-7112-45b1-af9c-542483c4f873 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:45.113666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:45.273356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62495e3d-85a8-4b39-880a-d5b9ac51cc65 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:45.283571 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7820a4-7112-45b1-af9c-542483c4f873 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:45.331344 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:45.346077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b8143-af6b-435b-84d0-ae551af2c2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:45.346699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b8143-af6b-435b-84d0-ae551af2c2d1 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:45.347363 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b8143-af6b-435b-84d0-ae551af2c2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:45.347643 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b8143-af6b-435b-84d0-ae551af2c2d1 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:45.348459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b8143-af6b-435b-84d0-ae551af2c2d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:45.349912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b8143-af6b-435b-84d0-ae551af2c2d1 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:45.375892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e7291-56fd-41f6-97e7-af8269585fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:45.376556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e7291-56fd-41f6-97e7-af8269585fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:45.376805 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e7291-56fd-41f6-97e7-af8269585fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:45.377078 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e7291-56fd-41f6-97e7-af8269585fa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:45.470999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 72/146] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:04:45 2022] POST /identity/v3/users => generated 283 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:04:45.495198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e7291-56fd-41f6-97e7-af8269585fa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:45.496758 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e7291-56fd-41f6-97e7-af8269585fa0 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:45.508260 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:45.656695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 75/147] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:04:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 11:04:45.662543 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b8143-af6b-435b-84d0-ae551af2c2d1 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:45.716230 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457e9bb-f72a-4712-a2af-b57195d2528f None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:45.716558 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457e9bb-f72a-4712-a2af-b57195d2528f None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:45.716789 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457e9bb-f72a-4712-a2af-b57195d2528f None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:45.718257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 76/148] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:04:45.724841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4965862-3cd9-4d4d-b5dc-005e989b28fb None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:45.725141 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4965862-3cd9-4d4d-b5dc-005e989b28fb None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:45.725411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4965862-3cd9-4d4d-b5dc-005e989b28fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:45.756308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-d4965862-3cd9-4d4d-b5dc-005e989b28fb None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:45.758256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d4965862-3cd9-4d4d-b5dc-005e989b28fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:45.846505 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b8143-af6b-435b-84d0-ae551af2c2d1 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:45.881580 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 77/149] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:45.883794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21731ce0-aecb-4dab-8c5f-ffc75ea03b42 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:45.884085 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21731ce0-aecb-4dab-8c5f-ffc75ea03b42 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:45.884360 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21731ce0-aecb-4dab-8c5f-ffc75ea03b42 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:45.885857 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 78/150] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:45.889012 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe5cff-600e-4a9e-b685-b189c743fed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:45.889806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe5cff-600e-4a9e-b685-b189c743fed6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:45.890088 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe5cff-600e-4a9e-b685-b189c743fed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:45.890442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe5cff-600e-4a9e-b685-b189c743fed6 None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:45.891659 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe5cff-600e-4a9e-b685-b189c743fed6 None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:45.891762 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac5299-f021-4c90-81b8-9922f21d0247 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:45.891992 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac5299-f021-4c90-81b8-9922f21d0247 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:45.892267 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac5299-f021-4c90-81b8-9922f21d0247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:45.893109 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe5cff-600e-4a9e-b685-b189c743fed6 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:45.895336 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 73/151] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:45 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:45.905950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe5cff-600e-4a9e-b685-b189c743fed6 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:45.921885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-fdac5299-f021-4c90-81b8-9922f21d0247 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:45.925040 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-fdac5299-f021-4c90-81b8-9922f21d0247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:45.971119 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:45.975532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:46.041364 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 79/152] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:46.060807 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe5cff-600e-4a9e-b685-b189c743fed6 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:46.106515 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bd24e-56b1-4f79-8e19-dbcdf39c3d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:46.107198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bd24e-56b1-4f79-8e19-dbcdf39c3d62 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:46.107498 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bd24e-56b1-4f79-8e19-dbcdf39c3d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:46.107777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bd24e-56b1-4f79-8e19-dbcdf39c3d62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:46.108522 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bd24e-56b1-4f79-8e19-dbcdf39c3d62 None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=ff6a8ae52b074b64be28a144288b4780, user.id=b83cd9cf7ce84815942f0d89b245aefe) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:46.108834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bd24e-56b1-4f79-8e19-dbcdf39c3d62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:46.110331 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bd24e-56b1-4f79-8e19-dbcdf39c3d62 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:46.117592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 74/153] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:04:45 2022] GET /identity/v3/role_assignments?user.id=b83cd9cf7ce84815942f0d89b245aefe&scope.project.id=ff6a8ae52b074b64be28a144288b4780&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:46.244400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac5299-f021-4c90-81b8-9922f21d0247 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:46.268173 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733a6e7-6f06-44e6-92ea-48922f998191 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:46.268489 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733a6e7-6f06-44e6-92ea-48922f998191 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:46.268736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733a6e7-6f06-44e6-92ea-48922f998191 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:46.270429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 75/154] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:46.289198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5242b-a655-4598-976f-88e37e09e006 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:46.289789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5242b-a655-4598-976f-88e37e09e006 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:46.289789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5242b-a655-4598-976f-88e37e09e006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:46.326043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b4f5242b-a655-4598-976f-88e37e09e006 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:46.327311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b4f5242b-a655-4598-976f-88e37e09e006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:46.437126 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac5299-f021-4c90-81b8-9922f21d0247 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:46.442184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 76/155] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:46.443056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5242b-a655-4598-976f-88e37e09e006 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:46.487466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bd165-9c8b-4ed2-89de-6f4e315b1840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:46.488482 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bd165-9c8b-4ed2-89de-6f4e315b1840 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:46.488961 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bd165-9c8b-4ed2-89de-6f4e315b1840 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:46.489441 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bd165-9c8b-4ed2-89de-6f4e315b1840 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:46.501599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:46.504675 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bd165-9c8b-4ed2-89de-6f4e315b1840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:46.509853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-1a6bd165-9c8b-4ed2-89de-6f4e315b1840 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:04:46.509853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:46.509853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:46.509853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:46.509853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:46.509853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:46.509853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:46.509853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:46.509853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:46.509853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:46.509853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:46.509853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:46.509853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:46.509853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:46.509853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:46.509853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:46.515579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:46.515579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:46.515579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:46.515579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:46.515579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:46.515579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:46.515579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:46.515579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:46.515579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:46.515579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:46.515579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:46.515579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:46.515579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:46.515579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:46.515579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:46.515579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:46.516539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:46.516539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:46.516539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:46.516539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:46.516539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:46.516539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:46.516539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:46.516539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:46.516539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:46.516539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:46.516539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:04:46.516539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:04:46.516539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 80/156] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:46.522116 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bd165-9c8b-4ed2-89de-6f4e315b1840 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:46.557200 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:46.568148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5242b-a655-4598-976f-88e37e09e006 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:46.608848 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f2ec9-f444-4987-a812-69855203c9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:46.609584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f2ec9-f444-4987-a812-69855203c9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:46.609877 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f2ec9-f444-4987-a812-69855203c9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:46.610152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f2ec9-f444-4987-a812-69855203c9b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:46.626794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f2ec9-f444-4987-a812-69855203c9b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:46.632458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-ca1f2ec9-f444-4987-a812-69855203c9b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:04:46.632458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:46.632458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:46.632458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:46.632458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:46.632458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:46.632458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:46.632458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:46.632458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:46.632458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:46.632458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:46.632458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:46.632458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:46.632458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:46.632458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:46.632458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:46.633385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:46.633385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:46.633385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:46.633385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:46.633385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:46.633385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:46.633385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:46.633385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:46.633385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:46.633385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:46.633385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:46.633385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:46.633385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:46.633385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:46.633385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:46.633385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:46.634157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:46.634157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:46.634157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:46.634157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:46.634157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:46.634157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:46.634157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:46.634157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:46.634157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:46.634157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:46.634157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:04:46.634157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:04:46.634157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 77/157] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:46.640472 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f2ec9-f444-4987-a812-69855203c9b2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:46.687541 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bd165-9c8b-4ed2-89de-6f4e315b1840 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:46.757128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f2ec9-f444-4987-a812-69855203c9b2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:46.758736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:46.785157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd154ce9-2027-477d-b327-d6b3fc62048b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:46.786095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd154ce9-2027-477d-b327-d6b3fc62048b None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:46.786579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd154ce9-2027-477d-b327-d6b3fc62048b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:46.787051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd154ce9-2027-477d-b327-d6b3fc62048b None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:46.788011 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd154ce9-2027-477d-b327-d6b3fc62048b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:46.788530 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd154ce9-2027-477d-b327-d6b3fc62048b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:46.790099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd154ce9-2027-477d-b327-d6b3fc62048b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:46.795806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef0057-26d2-4148-9612-e24f417989b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:46.796588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef0057-26d2-4148-9612-e24f417989b5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:46.796876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef0057-26d2-4148-9612-e24f417989b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:46.797156 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef0057-26d2-4148-9612-e24f417989b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:46.797956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef0057-26d2-4148-9612-e24f417989b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:46.798271 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef0057-26d2-4148-9612-e24f417989b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:46.799806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef0057-26d2-4148-9612-e24f417989b5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:46.810243 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 78/158] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:04:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:46.811824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 81/159] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:04:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:46.817491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef0057-26d2-4148-9612-e24f417989b5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:46.836199 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd154ce9-2027-477d-b327-d6b3fc62048b None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:46.963670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef0057-26d2-4148-9612-e24f417989b5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:47.004885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:47.010517 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad1fa8-f2b7-47c2-b9e7-a56e2bdcf977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:47.011241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad1fa8-f2b7-47c2-b9e7-a56e2bdcf977 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:47.011609 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad1fa8-f2b7-47c2-b9e7-a56e2bdcf977 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:47.011922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad1fa8-f2b7-47c2-b9e7-a56e2bdcf977 None admin] PATH_INFO: `/v3/users/97b48b02dddd451687de91f70bf306ee` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:47.016177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad1fa8-f2b7-47c2-b9e7-a56e2bdcf977 None admin] RBAC: Authorizing `identity:get_user(user_id=97b48b02dddd451687de91f70bf306ee)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:47.018038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad1fa8-f2b7-47c2-b9e7-a56e2bdcf977 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:47.025700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd154ce9-2027-477d-b327-d6b3fc62048b None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:47.025844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 79/160] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:46 2022] GET /identity/v3/users/97b48b02dddd451687de91f70bf306ee => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:47.025844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad1fa8-f2b7-47c2-b9e7-a56e2bdcf977 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:47.061800 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c652ca9-31dd-4f67-8504-326e00968a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:47.062451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c652ca9-31dd-4f67-8504-326e00968a1c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:47.062733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c652ca9-31dd-4f67-8504-326e00968a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:47.063014 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c652ca9-31dd-4f67-8504-326e00968a1c None admin] PATH_INFO: `/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:47.064330 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c652ca9-31dd-4f67-8504-326e00968a1c None admin] RBAC: Authorizing `identity:get_user(user_id=3cecb6cf43464e65ba7e69e9eb50ccbb)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:47.065815 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c652ca9-31dd-4f67-8504-326e00968a1c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:47.069181 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 82/161] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:46 2022] GET /identity/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:47.073921 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c652ca9-31dd-4f67-8504-326e00968a1c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:47.074978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:47.194329 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad1fa8-f2b7-47c2-b9e7-a56e2bdcf977 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:47.201783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c652ca9-31dd-4f67-8504-326e00968a1c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:47.257323 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794dfc7-339c-4ad5-bc14-c952332d2310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:47.257991 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794dfc7-339c-4ad5-bc14-c952332d2310 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:47.258270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794dfc7-339c-4ad5-bc14-c952332d2310 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:47.258539 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794dfc7-339c-4ad5-bc14-c952332d2310 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:47.268435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f481a-b498-49b7-a947-9aebfcb4063c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:47.269483 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f481a-b498-49b7-a947-9aebfcb4063c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:47.269967 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f481a-b498-49b7-a947-9aebfcb4063c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:47.270471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f481a-b498-49b7-a947-9aebfcb4063c None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:47.272089 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f481a-b498-49b7-a947-9aebfcb4063c None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:47.273906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f481a-b498-49b7-a947-9aebfcb4063c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:47.276238 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 83/162] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:47 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:47.280034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f481a-b498-49b7-a947-9aebfcb4063c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:47.289680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794dfc7-339c-4ad5-bc14-c952332d2310 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:47.296094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-6794dfc7-339c-4ad5-bc14-c952332d2310 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:04:47.296094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:47.296094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:47.296094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:47.296094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:47.296094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:47.296094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:47.296094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:47.296094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:47.296094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:47.296094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:47.296094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:47.296094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:47.296094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:47.296094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:47.296094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:47.297162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:47.297162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:47.297162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:47.297162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:47.297162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:47.297162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:47.297162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:47.297162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:47.297162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:47.297162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:47.297162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:47.297162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:47.297162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:47.297162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:47.297162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:47.297162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:47.297958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:47.297958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:47.297958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:47.297958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:47.297958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:47.297958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:47.297958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:47.297958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:47.297958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:47.297958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:47.297958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:04:47.297958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:04:47.304945 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 80/163] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:47.306549 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794dfc7-339c-4ad5-bc14-c952332d2310 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:47.335693 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:47.455583 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:47.469946 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794dfc7-339c-4ad5-bc14-c952332d2310 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:47.483052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f481a-b498-49b7-a947-9aebfcb4063c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:47.527182 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb706d-9fac-47f6-b215-101c49c59cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:47.527811 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb706d-9fac-47f6-b215-101c49c59cd6 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:47.528096 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb706d-9fac-47f6-b215-101c49c59cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:47.528372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb706d-9fac-47f6-b215-101c49c59cd6 None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780/users/97b48b02dddd451687de91f70bf306ee/roles/05303606367c4086bb8e3c0c20d45b56` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:47.532598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb706d-9fac-47f6-b215-101c49c59cd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff6a8ae52b074b64be28a144288b4780, user_id=97b48b02dddd451687de91f70bf306ee, role_id=05303606367c4086bb8e3c0c20d45b56)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:47.534066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb706d-9fac-47f6-b215-101c49c59cd6 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:47.549045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7f3c2-7df3-4e38-8e2c-2e3272629067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:47.557043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7f3c2-7df3-4e38-8e2c-2e3272629067 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:47.557043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7f3c2-7df3-4e38-8e2c-2e3272629067 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:47.557043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7f3c2-7df3-4e38-8e2c-2e3272629067 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:47.611201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 81/164] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:04:47 2022] PUT /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780/users/97b48b02dddd451687de91f70bf306ee/roles/05303606367c4086bb8e3c0c20d45b56 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:04:47.611201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb706d-9fac-47f6-b215-101c49c59cd6 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:47.671094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7f3c2-7df3-4e38-8e2c-2e3272629067 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:47.677282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7f3c2-7df3-4e38-8e2c-2e3272629067 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:47.747605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:47.830310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb706d-9fac-47f6-b215-101c49c59cd6 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:47.905015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fb69a-1493-4920-9083-bd7a1db18a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:47.905801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fb69a-1493-4920-9083-bd7a1db18a27 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:47.906084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fb69a-1493-4920-9083-bd7a1db18a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:47.906354 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fb69a-1493-4920-9083-bd7a1db18a27 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:47.907038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fb69a-1493-4920-9083-bd7a1db18a27 None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:47.907367 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fb69a-1493-4920-9083-bd7a1db18a27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:47.908768 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fb69a-1493-4920-9083-bd7a1db18a27 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:47.919481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 84/165] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:04:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:04:47.921738 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9d59d-75b3-40bf-b200-1288cd98dd2d None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:47.922026 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9d59d-75b3-40bf-b200-1288cd98dd2d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:47.922299 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9d59d-75b3-40bf-b200-1288cd98dd2d None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:47.928502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 85/166] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:47.929313 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 82/167] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:04:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:47.930786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9d59d-75b3-40bf-b200-1288cd98dd2d None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:47.933927 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc433908-d846-4b31-be69-9f4a237e2e5f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:47.934422 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc433908-d846-4b31-be69-9f4a237e2e5f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:47.934707 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc433908-d846-4b31-be69-9f4a237e2e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:47.963367 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-fc433908-d846-4b31-be69-9f4a237e2e5f None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:47.964977 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fc433908-d846-4b31-be69-9f4a237e2e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:47.978506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:48.061731 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 83/168] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:04:48.062794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc433908-d846-4b31-be69-9f4a237e2e5f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:48.155941 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9d59d-75b3-40bf-b200-1288cd98dd2d None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:48.209067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:48.229249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca94d75-1bd0-471b-ac83-93660677d2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:48.230440 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca94d75-1bd0-471b-ac83-93660677d2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:48.230713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca94d75-1bd0-471b-ac83-93660677d2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:48.230985 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca94d75-1bd0-471b-ac83-93660677d2b7 None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:48.235944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca94d75-1bd0-471b-ac83-93660677d2b7 None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:48.237416 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca94d75-1bd0-471b-ac83-93660677d2b7 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:48.239306 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 86/169] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:47 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:48.251876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca94d75-1bd0-471b-ac83-93660677d2b7 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:48.253656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc433908-d846-4b31-be69-9f4a237e2e5f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:48.323977 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abdfa92-d08f-450c-a4f2-57db3a0e62a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:48.324732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdfa92-d08f-450c-a4f2-57db3a0e62a7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:48.325028 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdfa92-d08f-450c-a4f2-57db3a0e62a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:48.325309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdfa92-d08f-450c-a4f2-57db3a0e62a7 None admin] PATH_INFO: `/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:48.327592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abdfa92-d08f-450c-a4f2-57db3a0e62a7 None admin] RBAC: Authorizing `identity:get_user(user_id=3cecb6cf43464e65ba7e69e9eb50ccbb)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:48.329063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abdfa92-d08f-450c-a4f2-57db3a0e62a7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:48.330853 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 84/170] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:48 2022] GET /identity/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb => generated 275 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:48.336531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abdfa92-d08f-450c-a4f2-57db3a0e62a7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:48.368579 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:48.444780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca94d75-1bd0-471b-ac83-93660677d2b7 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:48.513252 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd63aaf-b06a-4b8b-a92e-426d58424eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:48.513897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd63aaf-b06a-4b8b-a92e-426d58424eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:48.514137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd63aaf-b06a-4b8b-a92e-426d58424eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:48.514401 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd63aaf-b06a-4b8b-a92e-426d58424eb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:48.515132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd63aaf-b06a-4b8b-a92e-426d58424eb9 None admin] RBAC: Adding query filter params (role.id=486614b0ab4b454b92e9ef0bc7ec61c1, scope.project.id=ff6a8ae52b074b64be28a144288b4780, user.id=3cecb6cf43464e65ba7e69e9eb50ccbb) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:48.515466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd63aaf-b06a-4b8b-a92e-426d58424eb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:48.516903 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd63aaf-b06a-4b8b-a92e-426d58424eb9 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:48.528308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 87/171] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:04:48 2022] GET /identity/v3/role_assignments?user.id=3cecb6cf43464e65ba7e69e9eb50ccbb&scope.project.id=ff6a8ae52b074b64be28a144288b4780&role.id=486614b0ab4b454b92e9ef0bc7ec61c1 => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:48.598283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abdfa92-d08f-450c-a4f2-57db3a0e62a7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:48.659785 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6542f2-8dc6-4af0-8fdd-f606541f4378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:48.660341 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6542f2-8dc6-4af0-8fdd-f606541f4378 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:48.660627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6542f2-8dc6-4af0-8fdd-f606541f4378 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:48.660906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6542f2-8dc6-4af0-8fdd-f606541f4378 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:48.745337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd63aaf-b06a-4b8b-a92e-426d58424eb9 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:48.756632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6542f2-8dc6-4af0-8fdd-f606541f4378 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:48.758121 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6542f2-8dc6-4af0-8fdd-f606541f4378 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:48.780814 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:48.883512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:48.942250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd63aaf-b06a-4b8b-a92e-426d58424eb9 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:48.980552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 85/172] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:04:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:04:48.986334 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6542f2-8dc6-4af0-8fdd-f606541f4378 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:48.990108 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900b244-3367-4d74-af6e-51eaca53bbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:48.990754 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900b244-3367-4d74-af6e-51eaca53bbff None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:48.991029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900b244-3367-4d74-af6e-51eaca53bbff None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:48.991340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900b244-3367-4d74-af6e-51eaca53bbff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:49.014784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900b244-3367-4d74-af6e-51eaca53bbff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:49.021367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-e900b244-3367-4d74-af6e-51eaca53bbff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:04:49.021367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:49.021367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:49.021367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:49.021367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:49.021367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:49.021367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:49.021367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:49.021367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:49.021367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:49.021367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:49.021367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:49.021367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:49.021367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:49.021367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:49.021367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:49.022335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:49.022335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:49.022335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:49.022335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:49.022335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:49.022335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:49.022335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:49.022335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:49.022335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:49.022335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:49.022335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:49.022335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:49.022335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:49.022335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:49.022335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:49.022335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:49.023132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:49.023132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:49.023132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:49.023132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:49.023132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:49.023132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:49.023132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:49.023132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:49.023132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:49.023132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:49.023132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:04:49.023132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:04:49.023132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 88/173] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:04:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:04:49.034216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900b244-3367-4d74-af6e-51eaca53bbff None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:49.118312 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6542f2-8dc6-4af0-8fdd-f606541f4378 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:49.133241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:49.168894 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dae9f0-d780-4794-bbef-206be8653751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:49.169617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dae9f0-d780-4794-bbef-206be8653751 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:49.169898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dae9f0-d780-4794-bbef-206be8653751 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:49.170166 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dae9f0-d780-4794-bbef-206be8653751 None admin] PATH_INFO: `/v3/projects/b7499478eb9e4bd9bff19696f61ceff5` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:49.171337 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dae9f0-d780-4794-bbef-206be8653751 None admin] RBAC: Authorizing `identity:get_project(project_id=b7499478eb9e4bd9bff19696f61ceff5)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:49.172947 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dae9f0-d780-4794-bbef-206be8653751 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:49.174585 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 86/174] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:48 2022] GET /identity/v3/projects/b7499478eb9e4bd9bff19696f61ceff5 => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:49.180891 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dae9f0-d780-4794-bbef-206be8653751 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:49.305118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900b244-3367-4d74-af6e-51eaca53bbff None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:49.320811 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dae9f0-d780-4794-bbef-206be8653751 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:49.347537 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:49.372413 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb887e1-5305-4ea6-ab0c-0bf9657b8c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:49.373093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb887e1-5305-4ea6-ab0c-0bf9657b8c51 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:49.373316 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb887e1-5305-4ea6-ab0c-0bf9657b8c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:49.373588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb887e1-5305-4ea6-ab0c-0bf9657b8c51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:49.374326 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb887e1-5305-4ea6-ab0c-0bf9657b8c51 None admin] RBAC: Adding query filter params (role.id=486614b0ab4b454b92e9ef0bc7ec61c1, scope.project.id=b7499478eb9e4bd9bff19696f61ceff5, user.id=3cecb6cf43464e65ba7e69e9eb50ccbb) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:49.374628 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb887e1-5305-4ea6-ab0c-0bf9657b8c51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:49.376089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb887e1-5305-4ea6-ab0c-0bf9657b8c51 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:49.380564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ea626-74a3-474c-b577-ff7af1e7cab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:49.381191 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ea626-74a3-474c-b577-ff7af1e7cab9 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:49.381473 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ea626-74a3-474c-b577-ff7af1e7cab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:49.381750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ea626-74a3-474c-b577-ff7af1e7cab9 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:49.382429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ea626-74a3-474c-b577-ff7af1e7cab9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:49.382738 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ea626-74a3-474c-b577-ff7af1e7cab9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:49.383020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 87/175] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:04:49 2022] GET /identity/v3/role_assignments?user.id=3cecb6cf43464e65ba7e69e9eb50ccbb&scope.project.id=b7499478eb9e4bd9bff19696f61ceff5&role.id=486614b0ab4b454b92e9ef0bc7ec61c1 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:49.384165 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ea626-74a3-474c-b577-ff7af1e7cab9 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:49.403477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 89/176] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:04:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:49.405675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb887e1-5305-4ea6-ab0c-0bf9657b8c51 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:49.493634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:49.582082 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb887e1-5305-4ea6-ab0c-0bf9657b8c51 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:49.631381 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913db732-f7bb-4aa1-b8fb-6591cdf70198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:49.632034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913db732-f7bb-4aa1-b8fb-6591cdf70198 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:49.632320 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913db732-f7bb-4aa1-b8fb-6591cdf70198 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:49.632598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913db732-f7bb-4aa1-b8fb-6591cdf70198 None admin] PATH_INFO: `/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:49.640153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913db732-f7bb-4aa1-b8fb-6591cdf70198 None admin] RBAC: Authorizing `identity:get_user(user_id=3cecb6cf43464e65ba7e69e9eb50ccbb)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:49.641652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913db732-f7bb-4aa1-b8fb-6591cdf70198 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:49.643391 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 88/177] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:49 2022] GET /identity/v3/users/3cecb6cf43464e65ba7e69e9eb50ccbb => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:49.649010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ea626-74a3-474c-b577-ff7af1e7cab9 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:49.801597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ea626-74a3-474c-b577-ff7af1e7cab9 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:49.849965 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcca573-9178-47c5-961c-9126bcf3e577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:49.850973 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcca573-9178-47c5-961c-9126bcf3e577 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:49.851470 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcca573-9178-47c5-961c-9126bcf3e577 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:49.851963 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcca573-9178-47c5-961c-9126bcf3e577 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:49.914620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:49.935111 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcca573-9178-47c5-961c-9126bcf3e577 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:49.936601 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcca573-9178-47c5-961c-9126bcf3e577 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:50.003781 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39cc09-28c4-42b6-a850-7fd590f638e9 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:50.004687 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39cc09-28c4-42b6-a850-7fd590f638e9 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:50.005180 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39cc09-28c4-42b6-a850-7fd590f638e9 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:50.006979 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 89/178] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:04:50.015054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58023b-5d10-4694-9781-d7546898cacc None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:50.015663 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58023b-5d10-4694-9781-d7546898cacc None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:50.016129 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58023b-5d10-4694-9781-d7546898cacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:50.050222 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-5b58023b-5d10-4694-9781-d7546898cacc None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:50.054366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5b58023b-5d10-4694-9781-d7546898cacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:50.090278 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 90/179] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:04:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:04:50.096210 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcca573-9178-47c5-961c-9126bcf3e577 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:50.233599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 90/180] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:04:50.251581 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:50.270913 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcca573-9178-47c5-961c-9126bcf3e577 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:50.300682 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ba965-da9f-43b4-8235-f2016ffc098f None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:50.301000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ba965-da9f-43b4-8235-f2016ffc098f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:50.301245 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ba965-da9f-43b4-8235-f2016ffc098f None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:50.302803 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 91/181] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:04:50.308424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5aa784-dea2-478b-990f-8b125fb7d1e3 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:50.308712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5aa784-dea2-478b-990f-8b125fb7d1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:50.308993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5aa784-dea2-478b-990f-8b125fb7d1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:50.322725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061d414-c440-4c57-800b-4fea75d75d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:50.323969 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061d414-c440-4c57-800b-4fea75d75d92 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:50.324246 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061d414-c440-4c57-800b-4fea75d75d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:50.324522 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061d414-c440-4c57-800b-4fea75d75d92 None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:50.329506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-2a5aa784-dea2-478b-990f-8b125fb7d1e3 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:50.330931 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2a5aa784-dea2-478b-990f-8b125fb7d1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:50.331995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061d414-c440-4c57-800b-4fea75d75d92 None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:50.333606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061d414-c440-4c57-800b-4fea75d75d92 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:50.339889 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 91/182] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:50 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:50.345533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061d414-c440-4c57-800b-4fea75d75d92 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:50.432384 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:50.457529 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061d414-c440-4c57-800b-4fea75d75d92 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:50.461268 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 92/183] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:04:50.464828 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fde09-67a3-41ad-8a06-8b015ccc78c7 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:50.464828 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fde09-67a3-41ad-8a06-8b015ccc78c7 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:50.464828 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fde09-67a3-41ad-8a06-8b015ccc78c7 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:50.467885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 93/184] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:50.470149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddddc13-1f12-4a85-ad86-6c9afb0943af None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:50.470439 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddddc13-1f12-4a85-ad86-6c9afb0943af None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:50.470721 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddddc13-1f12-4a85-ad86-6c9afb0943af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:50.498285 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0ddddc13-1f12-4a85-ad86-6c9afb0943af None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:50.501690 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0ddddc13-1f12-4a85-ad86-6c9afb0943af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:50.535598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2e846-3596-4436-9416-09e8f4da6e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:50.536570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2e846-3596-4436-9416-09e8f4da6e31 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:50.536985 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2e846-3596-4436-9416-09e8f4da6e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:50.537264 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2e846-3596-4436-9416-09e8f4da6e31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:50.538032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2e846-3596-4436-9416-09e8f4da6e31 None admin] RBAC: Adding query filter params (role.id=b7eccd60e28d4b39b54879dad8230e0d, scope.project.id=ff6a8ae52b074b64be28a144288b4780, user.id=3cecb6cf43464e65ba7e69e9eb50ccbb) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:50.538344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2e846-3596-4436-9416-09e8f4da6e31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:50.539878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2e846-3596-4436-9416-09e8f4da6e31 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:50.549162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 92/185] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:04:50 2022] GET /identity/v3/role_assignments?user.id=3cecb6cf43464e65ba7e69e9eb50ccbb&scope.project.id=ff6a8ae52b074b64be28a144288b4780&role.id=b7eccd60e28d4b39b54879dad8230e0d => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:50.552104 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2e846-3596-4436-9416-09e8f4da6e31 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:50.593117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:50.647267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 94/186] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:04:50.738878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2e846-3596-4436-9416-09e8f4da6e31 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:50.782624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578feb18-fe2a-439c-adca-5cd3e0ec1824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:50.786150 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578feb18-fe2a-439c-adca-5cd3e0ec1824 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:50.786459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578feb18-fe2a-439c-adca-5cd3e0ec1824 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:50.786743 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578feb18-fe2a-439c-adca-5cd3e0ec1824 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:50.799412 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578feb18-fe2a-439c-adca-5cd3e0ec1824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:50.804797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-578feb18-fe2a-439c-adca-5cd3e0ec1824 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:04:50.804797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:50.804797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:50.804797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:50.804797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:50.804797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:50.804797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:50.804797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:50.804797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:50.804797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:50.804797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:50.804797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:50.804797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:50.804797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:50.804797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:50.804797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:50.805652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:50.805652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:50.805652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:50.805652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:50.805652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:50.805652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:50.805652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:50.805652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:50.805652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:50.805652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:50.805652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:50.805652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:50.805652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:50.805652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:50.805652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:50.805652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:50.806864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:50.806864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:50.806864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:50.806864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:50.806864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:50.806864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:50.806864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:50.806864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:50.806864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:50.806864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:50.806864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:04:50.806864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:04:50.806864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 93/187] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:04:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:50.812159 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddddc13-1f12-4a85-ad86-6c9afb0943af None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:50.826217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b7c2c-d0b4-4f04-804a-568d905f8d32 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:50.826501 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b7c2c-d0b4-4f04-804a-568d905f8d32 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:50.826774 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b7c2c-d0b4-4f04-804a-568d905f8d32 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:50.828465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 94/188] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:50.835980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df117e7-05df-4450-affa-d749d70c6c77 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:50.836269 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df117e7-05df-4450-affa-d749d70c6c77 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:50.836545 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df117e7-05df-4450-affa-d749d70c6c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:50.860847 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-4df117e7-05df-4450-affa-d749d70c6c77 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:50.862575 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4df117e7-05df-4450-affa-d749d70c6c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:50.938987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddddc13-1f12-4a85-ad86-6c9afb0943af None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:50.941695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:50.956509 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 95/189] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:50.957676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df117e7-05df-4450-affa-d749d70c6c77 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:50.963082 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:50.976901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabe2f5-be78-4ed8-86a4-c5a79c4e6bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:50.977532 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabe2f5-be78-4ed8-86a4-c5a79c4e6bdd None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:50.977784 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabe2f5-be78-4ed8-86a4-c5a79c4e6bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:50.978067 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabe2f5-be78-4ed8-86a4-c5a79c4e6bdd None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:50.978726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabe2f5-be78-4ed8-86a4-c5a79c4e6bdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:50.979023 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabe2f5-be78-4ed8-86a4-c5a79c4e6bdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:50.981029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabe2f5-be78-4ed8-86a4-c5a79c4e6bdd None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:50.994731 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 95/190] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:50.994731 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabe2f5-be78-4ed8-86a4-c5a79c4e6bdd None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:51.080303 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df117e7-05df-4450-affa-d749d70c6c77 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:51.100364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabe2f5-be78-4ed8-86a4-c5a79c4e6bdd None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:51.124840 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978f4b4-6a07-4def-b3fd-b3f15265429c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:51.125632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978f4b4-6a07-4def-b3fd-b3f15265429c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:51.125832 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978f4b4-6a07-4def-b3fd-b3f15265429c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:51.126106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978f4b4-6a07-4def-b3fd-b3f15265429c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:51.137829 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978f4b4-6a07-4def-b3fd-b3f15265429c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:51.140131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0d59e-741b-4cb4-b553-e89f90a3bf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:51.140792 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0d59e-741b-4cb4-b553-e89f90a3bf97 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:51.141066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0d59e-741b-4cb4-b553-e89f90a3bf97 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:51.143040 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0d59e-741b-4cb4-b553-e89f90a3bf97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:51.143040 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0d59e-741b-4cb4-b553-e89f90a3bf97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:51.143345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-8978f4b4-6a07-4def-b3fd-b3f15265429c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:04:51.143345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:51.143345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:51.143345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:51.143345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:51.143345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:51.143345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:51.143345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:51.143345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:51.143345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:51.143345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:51.143345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:51.143345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:51.143345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:51.143345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:51.143345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:51.144272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:51.144272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:51.144272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:51.144272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:51.144272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:51.144272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:51.144272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:51.144272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:51.144272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:51.144272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:51.144272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:51.144272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:51.144272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:51.144272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:51.144272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:51.144272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:51.145088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:51.145088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:51.145088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:51.145088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:51.145088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:51.145088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:51.145088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:51.145088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:51.145088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:51.145088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:51.145088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:04:51.145088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:04:51.145678 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0d59e-741b-4cb4-b553-e89f90a3bf97 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:51.145744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 96/191] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:04:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:51.145744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978f4b4-6a07-4def-b3fd-b3f15265429c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:51.150542 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 96/192] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:04:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:51.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0d59e-741b-4cb4-b553-e89f90a3bf97 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:51.186013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:51.220375 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:51.262024 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0d59e-741b-4cb4-b553-e89f90a3bf97 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:51.264781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978f4b4-6a07-4def-b3fd-b3f15265429c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:51.296994 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4af5f0-0ce0-47de-90f8-04279c7797dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:51.297614 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4af5f0-0ce0-47de-90f8-04279c7797dc None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:51.297903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4af5f0-0ce0-47de-90f8-04279c7797dc None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:51.298177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4af5f0-0ce0-47de-90f8-04279c7797dc None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:51.298849 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4af5f0-0ce0-47de-90f8-04279c7797dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:51.299154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4af5f0-0ce0-47de-90f8-04279c7797dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:51.299619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9df96e-59eb-4483-bb99-0a97247b5224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:51.300261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9df96e-59eb-4483-bb99-0a97247b5224 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:51.300628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9df96e-59eb-4483-bb99-0a97247b5224 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:51.300728 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4af5f0-0ce0-47de-90f8-04279c7797dc None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:51.301021 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9df96e-59eb-4483-bb99-0a97247b5224 None admin] PATH_INFO: `/v3/users/e9c5aabeb22c44c5a885cfbf39259d6b` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:51.308905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 97/193] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:04:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:51.308905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4af5f0-0ce0-47de-90f8-04279c7797dc None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:51.315413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9df96e-59eb-4483-bb99-0a97247b5224 None admin] RBAC: Authorizing `identity:get_user(user_id=e9c5aabeb22c44c5a885cfbf39259d6b)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:51.316902 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9df96e-59eb-4483-bb99-0a97247b5224 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:51.318577 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 97/194] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:51 2022] GET /identity/v3/users/e9c5aabeb22c44c5a885cfbf39259d6b => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:51.319372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9df96e-59eb-4483-bb99-0a97247b5224 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:51.411319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:51.417985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4af5f0-0ce0-47de-90f8-04279c7797dc None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:51.421753 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9df96e-59eb-4483-bb99-0a97247b5224 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:51.443312 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:51.453675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1674e9-9432-4655-850f-11aa9273cc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:51.454408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1674e9-9432-4655-850f-11aa9273cc8a None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:51.454573 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1674e9-9432-4655-850f-11aa9273cc8a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:51.454843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1674e9-9432-4655-850f-11aa9273cc8a None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:51.455612 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1674e9-9432-4655-850f-11aa9273cc8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:51.456980 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1674e9-9432-4655-850f-11aa9273cc8a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:51.457984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d287d9d-0b49-41b9-98f3-610259e032e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:51.458622 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d287d9d-0b49-41b9-98f3-610259e032e8 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:51.458902 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d287d9d-0b49-41b9-98f3-610259e032e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:51.459185 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d287d9d-0b49-41b9-98f3-610259e032e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:51.469518 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d287d9d-0b49-41b9-98f3-610259e032e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:51.474430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-9d287d9d-0b49-41b9-98f3-610259e032e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:04:51.474430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:51.474430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:51.474430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:51.474430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:51.474430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:51.474430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:51.474430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:51.474430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:51.474430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:51.474430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:51.474430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:51.474430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:51.474430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:51.474430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:51.474430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:51.475786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:51.475786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:51.475786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:51.475786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:51.475786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:51.475786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:51.475786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:51.475786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:51.475786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:51.475786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:51.475786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:51.475786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:51.475786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:51.475786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:51.475786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:51.475786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:51.476742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:51.476742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:51.476742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:51.476742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:51.476742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:51.476742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:51.476742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:51.476742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:51.476742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:51.476742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:51.476742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:04:51.476742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:04:51.476742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 98/195] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:51.476742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d287d9d-0b49-41b9-98f3-610259e032e8 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:51.543566 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 98/196] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:04:51 2022] POST /identity/v3/users => generated 297 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:04:51.544327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1674e9-9432-4655-850f-11aa9273cc8a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:51.596404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d287d9d-0b49-41b9-98f3-610259e032e8 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:51.635255 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2844c6a-51a8-4761-aae8-90292e818ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:51.636499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2844c6a-51a8-4761-aae8-90292e818ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:51.636774 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2844c6a-51a8-4761-aae8-90292e818ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:51.637114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2844c6a-51a8-4761-aae8-90292e818ec4 None admin] PATH_INFO: `/v3/users/e9c5aabeb22c44c5a885cfbf39259d6b` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:51.638662 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2844c6a-51a8-4761-aae8-90292e818ec4 None admin] RBAC: Authorizing `identity:get_user(user_id=e9c5aabeb22c44c5a885cfbf39259d6b)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:51.639897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2844c6a-51a8-4761-aae8-90292e818ec4 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:51.647057 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 99/197] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:51 2022] GET /identity/v3/users/e9c5aabeb22c44c5a885cfbf39259d6b => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:51.647057 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2844c6a-51a8-4761-aae8-90292e818ec4 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:51.680649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1674e9-9432-4655-850f-11aa9273cc8a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:51.706671 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:51.725651 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2368c3-1317-44c3-8525-75bfba11827c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:51.726296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2368c3-1317-44c3-8525-75bfba11827c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:51.726572 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2368c3-1317-44c3-8525-75bfba11827c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:51.726846 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2368c3-1317-44c3-8525-75bfba11827c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:51.751763 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:51.765469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2844c6a-51a8-4761-aae8-90292e818ec4 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:51.807457 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2368c3-1317-44c3-8525-75bfba11827c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:51.809282 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2368c3-1317-44c3-8525-75bfba11827c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:51.812634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650238a-d8d0-4463-93d0-289c96c1f4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:51.813251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650238a-d8d0-4463-93d0-289c96c1f4cd None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:51.813534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650238a-d8d0-4463-93d0-289c96c1f4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:51.813802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650238a-d8d0-4463-93d0-289c96c1f4cd None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:51.814475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650238a-d8d0-4463-93d0-289c96c1f4cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:51.814776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650238a-d8d0-4463-93d0-289c96c1f4cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:51.816195 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650238a-d8d0-4463-93d0-289c96c1f4cd None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:51.830118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 100/198] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:04:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:51.830998 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650238a-d8d0-4463-93d0-289c96c1f4cd None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:51.957515 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 99/199] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:04:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:04:51.958526 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2368c3-1317-44c3-8525-75bfba11827c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:51.969310 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650238a-d8d0-4463-93d0-289c96c1f4cd None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:51.982243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:52.008121 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520bc9b-d738-49d9-a793-5c4781263998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:52.008744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520bc9b-d738-49d9-a793-5c4781263998 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:52.009025 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520bc9b-d738-49d9-a793-5c4781263998 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:52.009298 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520bc9b-d738-49d9-a793-5c4781263998 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:52.020338 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:52.086565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2368c3-1317-44c3-8525-75bfba11827c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:52.095164 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520bc9b-d738-49d9-a793-5c4781263998 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:52.097350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520bc9b-d738-49d9-a793-5c4781263998 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:52.132796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39708248-b297-4180-9c65-9763493cdc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:52.133430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39708248-b297-4180-9c65-9763493cdc7e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:52.135564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39708248-b297-4180-9c65-9763493cdc7e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:52.135564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39708248-b297-4180-9c65-9763493cdc7e None admin] PATH_INFO: `/v3/users/97b48b02dddd451687de91f70bf306ee` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:52.135564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39708248-b297-4180-9c65-9763493cdc7e None admin] RBAC: Authorizing `identity:get_user(user_id=97b48b02dddd451687de91f70bf306ee)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:52.137296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39708248-b297-4180-9c65-9763493cdc7e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:52.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 100/200] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:51 2022] GET /identity/v3/users/97b48b02dddd451687de91f70bf306ee => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:52.139640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39708248-b297-4180-9c65-9763493cdc7e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:52.238448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 101/201] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:04:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:04:52.239641 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520bc9b-d738-49d9-a793-5c4781263998 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:52.259603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:52.268278 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39708248-b297-4180-9c65-9763493cdc7e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:52.309751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d191c8-bf5c-4884-86b8-cd74f28099be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:52.310528 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d191c8-bf5c-4884-86b8-cd74f28099be None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:52.310881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d191c8-bf5c-4884-86b8-cd74f28099be None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:52.311243 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d191c8-bf5c-4884-86b8-cd74f28099be None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:52.312523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d191c8-bf5c-4884-86b8-cd74f28099be None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:52.314245 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d191c8-bf5c-4884-86b8-cd74f28099be None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:52.317012 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 101/202] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:52 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:52.323024 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d191c8-bf5c-4884-86b8-cd74f28099be None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:52.326506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:52.354547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520bc9b-d738-49d9-a793-5c4781263998 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:52.396548 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b5527-5581-41e3-9be7-2bfea59d618d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:52.397515 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b5527-5581-41e3-9be7-2bfea59d618d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:52.397975 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b5527-5581-41e3-9be7-2bfea59d618d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:52.398426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b5527-5581-41e3-9be7-2bfea59d618d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:52.441782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d191c8-bf5c-4884-86b8-cd74f28099be None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:52.471595 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b5527-5581-41e3-9be7-2bfea59d618d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:52.473408 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b5527-5581-41e3-9be7-2bfea59d618d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:52.481552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9bade-295b-4275-a7c1-891f97b0a61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:52.482280 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9bade-295b-4275-a7c1-891f97b0a61c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:52.482626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9bade-295b-4275-a7c1-891f97b0a61c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:52.482964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9bade-295b-4275-a7c1-891f97b0a61c None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:52.484677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9bade-295b-4275-a7c1-891f97b0a61c None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:52.486546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9bade-295b-4275-a7c1-891f97b0a61c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:52.488504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:52.489607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 102/203] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:52 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:52.494717 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9bade-295b-4275-a7c1-891f97b0a61c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:52.579634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:52.603528 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 102/204] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:04:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:04:52.604238 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b5527-5581-41e3-9be7-2bfea59d618d None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:52.604546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9bade-295b-4275-a7c1-891f97b0a61c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:52.644845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52d066-22e8-4630-acca-ed7564e206eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:52.645927 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52d066-22e8-4630-acca-ed7564e206eb None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:52.646411 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52d066-22e8-4630-acca-ed7564e206eb None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:52.646883 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52d066-22e8-4630-acca-ed7564e206eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:52.647850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52d066-22e8-4630-acca-ed7564e206eb None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=1453ab88d2354019a189c0e0883fd8b1, user.id=e9c5aabeb22c44c5a885cfbf39259d6b) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:52.648476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52d066-22e8-4630-acca-ed7564e206eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:52.650689 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52d066-22e8-4630-acca-ed7564e206eb None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:52.657639 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 103/205] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:04:52 2022] GET /identity/v3/role_assignments?user.id=e9c5aabeb22c44c5a885cfbf39259d6b&scope.project.id=1453ab88d2354019a189c0e0883fd8b1&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:52.658765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52d066-22e8-4630-acca-ed7564e206eb None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:52.788254 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b5527-5581-41e3-9be7-2bfea59d618d None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:52.803447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:52.854071 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0683d3a-aacd-4e56-abb7-b1a8817d797c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:52.855144 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0683d3a-aacd-4e56-abb7-b1a8817d797c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:52.855658 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0683d3a-aacd-4e56-abb7-b1a8817d797c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:52.856154 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0683d3a-aacd-4e56-abb7-b1a8817d797c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:52.857126 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0683d3a-aacd-4e56-abb7-b1a8817d797c None admin] RBAC: Adding query filter params (role.id=b7eccd60e28d4b39b54879dad8230e0d, scope.project.id=1453ab88d2354019a189c0e0883fd8b1, user.id=e9c5aabeb22c44c5a885cfbf39259d6b) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:52.857660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0683d3a-aacd-4e56-abb7-b1a8817d797c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:52.859390 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0683d3a-aacd-4e56-abb7-b1a8817d797c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:52.866338 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 103/206] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:04:52 2022] GET /identity/v3/role_assignments?user.id=e9c5aabeb22c44c5a885cfbf39259d6b&scope.project.id=1453ab88d2354019a189c0e0883fd8b1&role.id=b7eccd60e28d4b39b54879dad8230e0d => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:52.874707 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52d066-22e8-4630-acca-ed7564e206eb None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:52.907680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:52.917057 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68c401-e596-48b1-ba25-f734d5139310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:52.920056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68c401-e596-48b1-ba25-f734d5139310 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:52.920056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68c401-e596-48b1-ba25-f734d5139310 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:52.920056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68c401-e596-48b1-ba25-f734d5139310 None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:52.920056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68c401-e596-48b1-ba25-f734d5139310 None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:52.921177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68c401-e596-48b1-ba25-f734d5139310 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:52.923904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 104/207] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:52 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:52.930063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0683d3a-aacd-4e56-abb7-b1a8817d797c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:53.087102 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0683d3a-aacd-4e56-abb7-b1a8817d797c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:53.134919 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc28fb-bfc0-4f28-a37b-5bb9f3a152c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:53.143549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc28fb-bfc0-4f28-a37b-5bb9f3a152c5 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:53.143549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc28fb-bfc0-4f28-a37b-5bb9f3a152c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:53.143549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc28fb-bfc0-4f28-a37b-5bb9f3a152c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:53.143549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc28fb-bfc0-4f28-a37b-5bb9f3a152c5 None admin] RBAC: Adding query filter params (role.id=05303606367c4086bb8e3c0c20d45b56, scope.project.id=ff6a8ae52b074b64be28a144288b4780, user.id=97b48b02dddd451687de91f70bf306ee) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:53.143549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc28fb-bfc0-4f28-a37b-5bb9f3a152c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:53.143549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc28fb-bfc0-4f28-a37b-5bb9f3a152c5 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:53.145848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 104/208] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:04:52 2022] GET /identity/v3/role_assignments?user.id=97b48b02dddd451687de91f70bf306ee&scope.project.id=ff6a8ae52b074b64be28a144288b4780&role.id=05303606367c4086bb8e3c0c20d45b56 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:54.058272 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051b648-3ccf-43e9-80b6-6e61fedee104 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:54.058778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051b648-3ccf-43e9-80b6-6e61fedee104 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:54.058882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051b648-3ccf-43e9-80b6-6e61fedee104 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:54.060217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 105/209] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:54.065735 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaef7e2-cbc4-490b-b274-19e83b7be233 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:54.066564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaef7e2-cbc4-490b-b274-19e83b7be233 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:54.067038 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaef7e2-cbc4-490b-b274-19e83b7be233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:54.086903 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-abaef7e2-cbc4-490b-b274-19e83b7be233 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:54.088474 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-abaef7e2-cbc4-490b-b274-19e83b7be233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:54.134147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ec0f6-98e7-48d0-8a54-dd5b6e9842bc None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:54.134475 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ec0f6-98e7-48d0-8a54-dd5b6e9842bc None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:54.134694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ec0f6-98e7-48d0-8a54-dd5b6e9842bc None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:54.135244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:54.136730 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 106/210] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:04:54.141931 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba1dd0-6519-43ec-99d8-19102d0664c5 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:54.142236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba1dd0-6519-43ec-99d8-19102d0664c5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:54.142473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba1dd0-6519-43ec-99d8-19102d0664c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:54.162177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-34ba1dd0-6519-43ec-99d8-19102d0664c5 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:54.163392 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-34ba1dd0-6519-43ec-99d8-19102d0664c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:54.179125 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 105/211] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:54.232640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 107/212] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:04:54.455907 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaef7e2-cbc4-490b-b274-19e83b7be233 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:54.502535 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba1dd0-6519-43ec-99d8-19102d0664c5 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:54.564461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaef7e2-cbc4-490b-b274-19e83b7be233 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:54.601287 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672a93b-2d3f-49c8-947f-1b46c5be1d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:54.601925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672a93b-2d3f-49c8-947f-1b46c5be1d49 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:54.602201 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672a93b-2d3f-49c8-947f-1b46c5be1d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:54.602467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672a93b-2d3f-49c8-947f-1b46c5be1d49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:54.603690 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672a93b-2d3f-49c8-947f-1b46c5be1d49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:54.605296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672a93b-2d3f-49c8-947f-1b46c5be1d49 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:54.607263 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 106/213] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:04:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:54.609915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba1dd0-6519-43ec-99d8-19102d0664c5 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:54.612464 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672a93b-2d3f-49c8-947f-1b46c5be1d49 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:54.629148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:54.650240 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e02266-2374-424c-a66c-1d3a9c48f6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:54.651196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:54.651843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e02266-2374-424c-a66c-1d3a9c48f6bd None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:54.652127 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e02266-2374-424c-a66c-1d3a9c48f6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:54.652399 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e02266-2374-424c-a66c-1d3a9c48f6bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:54.663744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e02266-2374-424c-a66c-1d3a9c48f6bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:54.669822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-30e02266-2374-424c-a66c-1d3a9c48f6bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:04:54.669822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:54.669822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:54.669822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:54.669822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:54.669822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:54.669822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:54.669822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:54.669822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:54.669822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:54.669822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:54.669822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:54.669822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:54.669822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:54.669822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:54.669822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:54.670696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:54.670696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:54.670696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:54.670696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:54.670696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:54.670696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:54.670696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:54.670696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:54.670696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:54.670696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:54.670696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:54.670696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:54.670696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:54.670696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:54.670696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:54.670696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:54.672322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:54.672322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:54.672322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:54.672322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:54.672322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:54.672322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:54.672322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:54.672322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:54.672322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:54.672322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:54.672322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:04:54.672322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:04:54.672322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 108/214] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:04:54.676989 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e02266-2374-424c-a66c-1d3a9c48f6bd None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:54.723480 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672a93b-2d3f-49c8-947f-1b46c5be1d49 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:54.761994 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037ea2a-e713-4494-8730-6284aa57df99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:54.762704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037ea2a-e713-4494-8730-6284aa57df99 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:54.762980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037ea2a-e713-4494-8730-6284aa57df99 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:54.763279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037ea2a-e713-4494-8730-6284aa57df99 None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:54.763930 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037ea2a-e713-4494-8730-6284aa57df99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:54.765509 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037ea2a-e713-4494-8730-6284aa57df99 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:54.784839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e02266-2374-424c-a66c-1d3a9c48f6bd None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:54.821990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5f77f-8116-47e5-a34e-87b3c784017a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:54.822677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5f77f-8116-47e5-a34e-87b3c784017a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:54.823017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5f77f-8116-47e5-a34e-87b3c784017a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:54.823999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5f77f-8116-47e5-a34e-87b3c784017a None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:54.824728 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5f77f-8116-47e5-a34e-87b3c784017a None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:54.825087 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5f77f-8116-47e5-a34e-87b3c784017a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:54.826615 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5f77f-8116-47e5-a34e-87b3c784017a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:54.838227 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 109/215] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:04:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:54.844101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5f77f-8116-47e5-a34e-87b3c784017a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:54.852947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 107/216] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:04:54 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:04:54.903767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:54.983160 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5f77f-8116-47e5-a34e-87b3c784017a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:55.028250 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed88b3f-e72d-4fdf-8a66-246046779fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:55.029295 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed88b3f-e72d-4fdf-8a66-246046779fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:55.029759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed88b3f-e72d-4fdf-8a66-246046779fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:55.030206 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed88b3f-e72d-4fdf-8a66-246046779fd9 None admin] PATH_INFO: `/v3/users/22b962825ca243418d673141782bec06` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:55.047022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed88b3f-e72d-4fdf-8a66-246046779fd9 None admin] RBAC: Authorizing `identity:get_user(user_id=22b962825ca243418d673141782bec06)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:55.049016 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed88b3f-e72d-4fdf-8a66-246046779fd9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:55.051401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 110/217] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:54 2022] GET /identity/v3/users/22b962825ca243418d673141782bec06 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:55.060070 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037ea2a-e713-4494-8730-6284aa57df99 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:55.097548 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c90a1-6e9f-4216-be20-d02f65f9d874 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:55.098049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c90a1-6e9f-4216-be20-d02f65f9d874 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:55.098145 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c90a1-6e9f-4216-be20-d02f65f9d874 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:55.099512 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 111/218] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:04:55.105669 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90303645-3ec3-4e47-98bb-ceb4f1c86c50 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:55.111933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90303645-3ec3-4e47-98bb-ceb4f1c86c50 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:55.112205 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90303645-3ec3-4e47-98bb-ceb4f1c86c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:55.140297 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-90303645-3ec3-4e47-98bb-ceb4f1c86c50 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:55.141748 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-90303645-3ec3-4e47-98bb-ceb4f1c86c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:55.179503 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:55.194045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037ea2a-e713-4494-8730-6284aa57df99 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:55.240407 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 112/219] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:55.242470 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e241c-7537-4b7f-a408-1d2aed3ccc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:55.243417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e241c-7537-4b7f-a408-1d2aed3ccc20 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:55.243922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e241c-7537-4b7f-a408-1d2aed3ccc20 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:55.244444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e241c-7537-4b7f-a408-1d2aed3ccc20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:55.328074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e241c-7537-4b7f-a408-1d2aed3ccc20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:55.329586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e241c-7537-4b7f-a408-1d2aed3ccc20 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:55.334735 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d348e5-e32d-441c-94e0-19e9bd122ff1 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:55.335050 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d348e5-e32d-441c-94e0-19e9bd122ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:55.335346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d348e5-e32d-441c-94e0-19e9bd122ff1 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:55.336950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 113/220] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:55 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:55.345366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43487d5-2f47-4c4b-905f-d1bfae3f0de5 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:55.345650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43487d5-2f47-4c4b-905f-d1bfae3f0de5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:55.345920 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43487d5-2f47-4c4b-905f-d1bfae3f0de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:55.369375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e43487d5-2f47-4c4b-905f-d1bfae3f0de5 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:55.371127 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e43487d5-2f47-4c4b-905f-d1bfae3f0de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:55.397210 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:55.440416 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:55.450916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 108/221] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:04:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:04:55.456743 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e241c-7537-4b7f-a408-1d2aed3ccc20 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:55.462942 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 114/222] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:55.582632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e241c-7537-4b7f-a408-1d2aed3ccc20 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:55.583477 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43487d5-2f47-4c4b-905f-d1bfae3f0de5 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:55.648485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacc5ee-eae2-4411-898f-d37a05ace2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:55.649139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacc5ee-eae2-4411-898f-d37a05ace2ff None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:55.649419 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacc5ee-eae2-4411-898f-d37a05ace2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:55.649691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacc5ee-eae2-4411-898f-d37a05ace2ff None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:55.653133 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacc5ee-eae2-4411-898f-d37a05ace2ff None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:55.654634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacc5ee-eae2-4411-898f-d37a05ace2ff None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:55.656366 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 109/223] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:55 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:55.662851 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacc5ee-eae2-4411-898f-d37a05ace2ff None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:55.765038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43487d5-2f47-4c4b-905f-d1bfae3f0de5 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:55.784315 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:55.831138 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63916d8-d7cd-47f1-b7f2-1ba05844b992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:55.832259 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63916d8-d7cd-47f1-b7f2-1ba05844b992 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:55.832922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63916d8-d7cd-47f1-b7f2-1ba05844b992 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:55.833226 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63916d8-d7cd-47f1-b7f2-1ba05844b992 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:55.842342 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacc5ee-eae2-4411-898f-d37a05ace2ff None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:55.845769 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63916d8-d7cd-47f1-b7f2-1ba05844b992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:55.850835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-c63916d8-d7cd-47f1-b7f2-1ba05844b992 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:04:55.850835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:55.850835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:55.850835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:55.850835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:55.850835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:55.850835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:55.850835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:55.850835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:55.850835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:55.850835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:55.850835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:55.850835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:55.850835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:55.850835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:55.850835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:55.851852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:55.851852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:55.851852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:55.851852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:55.851852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:55.851852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:55.851852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:55.851852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:55.851852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:55.851852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:55.851852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:55.851852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:55.851852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:55.851852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:55.851852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:55.851852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:55.852646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:55.852646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:55.852646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:55.852646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:55.852646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:55.852646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:55.852646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:55.852646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:55.852646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:55.852646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:55.852646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:04:55.852646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:04:55.852646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 115/224] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:04:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:04:55.853803 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63916d8-d7cd-47f1-b7f2-1ba05844b992 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:55.875971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:55.887569 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ddc79-bf37-421a-a895-2fafb55ca4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:55.888184 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ddc79-bf37-421a-a895-2fafb55ca4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:55.888457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ddc79-bf37-421a-a895-2fafb55ca4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:55.888720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ddc79-bf37-421a-a895-2fafb55ca4c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:55.889443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ddc79-bf37-421a-a895-2fafb55ca4c0 None admin] RBAC: Adding query filter params (role.id=486614b0ab4b454b92e9ef0bc7ec61c1, scope.project.id=1453ab88d2354019a189c0e0883fd8b1, user.id=22b962825ca243418d673141782bec06) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:55.889747 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ddc79-bf37-421a-a895-2fafb55ca4c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:55.891194 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ddc79-bf37-421a-a895-2fafb55ca4c0 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:55.903597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 110/225] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:04:55 2022] GET /identity/v3/role_assignments?user.id=22b962825ca243418d673141782bec06&scope.project.id=1453ab88d2354019a189c0e0883fd8b1&role.id=486614b0ab4b454b92e9ef0bc7ec61c1 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:55.904411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ddc79-bf37-421a-a895-2fafb55ca4c0 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:56.024204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63916d8-d7cd-47f1-b7f2-1ba05844b992 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:56.085001 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8ac5e-1c02-4542-a071-e4e39f130910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:56.085569 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8ac5e-1c02-4542-a071-e4e39f130910 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:56.085851 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8ac5e-1c02-4542-a071-e4e39f130910 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:56.086120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8ac5e-1c02-4542-a071-e4e39f130910 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:56.099307 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8ac5e-1c02-4542-a071-e4e39f130910 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:56.105153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-31a8ac5e-1c02-4542-a071-e4e39f130910 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:04:56.105153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:56.105153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:56.105153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:56.105153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:56.105153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:56.105153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:56.105153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:56.105153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:56.105153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:56.105153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:56.105153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:56.105153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:56.105153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:56.105153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:56.105153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:56.106126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:56.106126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:56.106126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:56.106126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:56.106126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:56.106126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:56.106126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:56.106126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:56.106126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:56.106126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:56.106126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:56.106126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:56.106126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:56.106126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:56.106126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:56.106126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:56.106915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:56.106915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:56.106915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:56.106915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:56.106915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:56.106915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:56.106915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:56.106915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:56.106915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:56.106915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:56.106915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:04:56.106915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:04:56.106915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 116/226] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:04:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:56.113871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8ac5e-1c02-4542-a071-e4e39f130910 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:56.127516 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ddc79-bf37-421a-a895-2fafb55ca4c0 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:56.155211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:56.175412 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e08254-eaf8-4ff6-83d2-9a6ca792087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:56.178087 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e08254-eaf8-4ff6-83d2-9a6ca792087c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:56.178608 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e08254-eaf8-4ff6-83d2-9a6ca792087c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:56.179082 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e08254-eaf8-4ff6-83d2-9a6ca792087c None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:56.181225 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e08254-eaf8-4ff6-83d2-9a6ca792087c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:56.184046 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e08254-eaf8-4ff6-83d2-9a6ca792087c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:56.186289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e08254-eaf8-4ff6-83d2-9a6ca792087c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:56.210253 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 111/227] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:56.214958 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e08254-eaf8-4ff6-83d2-9a6ca792087c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:56.263128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:56.313533 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8ac5e-1c02-4542-a071-e4e39f130910 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:56.381023 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e08254-eaf8-4ff6-83d2-9a6ca792087c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:56.390489 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35664acd-961c-4e98-8aa7-d3da6e51944f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:56.391121 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35664acd-961c-4e98-8aa7-d3da6e51944f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:56.391434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35664acd-961c-4e98-8aa7-d3da6e51944f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:56.391707 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35664acd-961c-4e98-8aa7-d3da6e51944f None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:56.392382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35664acd-961c-4e98-8aa7-d3da6e51944f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:56.392686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35664acd-961c-4e98-8aa7-d3da6e51944f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:56.394426 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35664acd-961c-4e98-8aa7-d3da6e51944f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:56.415214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 117/228] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:04:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:56.421030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35664acd-961c-4e98-8aa7-d3da6e51944f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:56.422911 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ed0cd-12b8-4c39-a4e4-f0d2db27f43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:56.423853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ed0cd-12b8-4c39-a4e4-f0d2db27f43c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:56.424335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ed0cd-12b8-4c39-a4e4-f0d2db27f43c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:56.424805 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ed0cd-12b8-4c39-a4e4-f0d2db27f43c None admin] PATH_INFO: `/v3/users/e9c5aabeb22c44c5a885cfbf39259d6b` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:56.426330 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ed0cd-12b8-4c39-a4e4-f0d2db27f43c None admin] RBAC: Authorizing `identity:get_user(user_id=e9c5aabeb22c44c5a885cfbf39259d6b)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:56.428069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ed0cd-12b8-4c39-a4e4-f0d2db27f43c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:56.430253 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 112/229] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:56 2022] GET /identity/v3/users/e9c5aabeb22c44c5a885cfbf39259d6b => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:56.436112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ed0cd-12b8-4c39-a4e4-f0d2db27f43c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:56.530103 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:56.569483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35664acd-961c-4e98-8aa7-d3da6e51944f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:56.618800 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ed0cd-12b8-4c39-a4e4-f0d2db27f43c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:56.644376 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93531b-dc99-4008-8f59-618016fd9c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:56.644948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93531b-dc99-4008-8f59-618016fd9c17 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:56.645231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93531b-dc99-4008-8f59-618016fd9c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:56.645509 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93531b-dc99-4008-8f59-618016fd9c17 None admin] PATH_INFO: `/v3/users/e9c5aabeb22c44c5a885cfbf39259d6b` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:56.647503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93531b-dc99-4008-8f59-618016fd9c17 None admin] RBAC: Authorizing `identity:get_user(user_id=e9c5aabeb22c44c5a885cfbf39259d6b)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:56.648961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93531b-dc99-4008-8f59-618016fd9c17 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:56.653267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 118/230] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:56 2022] GET /identity/v3/users/e9c5aabeb22c44c5a885cfbf39259d6b => generated 283 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:04:56.660660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93531b-dc99-4008-8f59-618016fd9c17 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:56.676866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:56.695518 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511e6a4-ef4b-4008-bd5c-1bf55bc36e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:56.696405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511e6a4-ef4b-4008-bd5c-1bf55bc36e82 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:56.696854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511e6a4-ef4b-4008-bd5c-1bf55bc36e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:56.697262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511e6a4-ef4b-4008-bd5c-1bf55bc36e82 None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:56.700181 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1511e6a4-ef4b-4008-bd5c-1bf55bc36e82 None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:56.701792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1511e6a4-ef4b-4008-bd5c-1bf55bc36e82 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:56.707433 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 113/231] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:56 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:56.716141 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511e6a4-ef4b-4008-bd5c-1bf55bc36e82 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:56.836858 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93531b-dc99-4008-8f59-618016fd9c17 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:56.869479 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511e6a4-ef4b-4008-bd5c-1bf55bc36e82 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:56.870869 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34234d94-e297-4f9c-9f87-96ca66baddf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:56.871541 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34234d94-e297-4f9c-9f87-96ca66baddf9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:56.871825 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34234d94-e297-4f9c-9f87-96ca66baddf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:56.872329 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34234d94-e297-4f9c-9f87-96ca66baddf9 None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:56.873563 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34234d94-e297-4f9c-9f87-96ca66baddf9 None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:56.875105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34234d94-e297-4f9c-9f87-96ca66baddf9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:56.878116 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 119/232] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:56 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:56.887288 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34234d94-e297-4f9c-9f87-96ca66baddf9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:56.908019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:56.927789 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc605d6-4c1f-44e0-99f7-7f002ee820c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:56.928776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc605d6-4c1f-44e0-99f7-7f002ee820c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:56.929199 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc605d6-4c1f-44e0-99f7-7f002ee820c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:56.929642 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc605d6-4c1f-44e0-99f7-7f002ee820c1 None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1/users/e9c5aabeb22c44c5a885cfbf39259d6b/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:56.936676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc605d6-4c1f-44e0-99f7-7f002ee820c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1453ab88d2354019a189c0e0883fd8b1, user_id=e9c5aabeb22c44c5a885cfbf39259d6b, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:56.938325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc605d6-4c1f-44e0-99f7-7f002ee820c1 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:56.987597 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:57.012897 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34234d94-e297-4f9c-9f87-96ca66baddf9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:57.013109 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 114/233] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:04:56 2022] PUT /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1/users/e9c5aabeb22c44c5a885cfbf39259d6b/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:04:57.064135 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374f70c-2471-47ec-b690-80aabe9a24e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:57.064819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374f70c-2471-47ec-b690-80aabe9a24e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:57.065095 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374f70c-2471-47ec-b690-80aabe9a24e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:57.065365 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374f70c-2471-47ec-b690-80aabe9a24e6 None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1/users/e9c5aabeb22c44c5a885cfbf39259d6b/roles/b7eccd60e28d4b39b54879dad8230e0d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:57.067672 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1374f70c-2471-47ec-b690-80aabe9a24e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=1453ab88d2354019a189c0e0883fd8b1, user_id=e9c5aabeb22c44c5a885cfbf39259d6b, role_id=b7eccd60e28d4b39b54879dad8230e0d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:57.069167 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1374f70c-2471-47ec-b690-80aabe9a24e6 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:57.152773 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 120/234] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:04:56 2022] PUT /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1/users/e9c5aabeb22c44c5a885cfbf39259d6b/roles/b7eccd60e28d4b39b54879dad8230e0d => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:04:57.663954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce5493-3513-4908-8cc4-1b3d9b2ef544 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:57.664506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce5493-3513-4908-8cc4-1b3d9b2ef544 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:57.664506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce5493-3513-4908-8cc4-1b3d9b2ef544 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:57.665948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 115/235] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:57.672197 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e18316-f01c-4d4d-a9b2-76877a99b20a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:57.672488 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e18316-f01c-4d4d-a9b2-76877a99b20a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:57.672763 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e18316-f01c-4d4d-a9b2-76877a99b20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:57.691545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-81e18316-f01c-4d4d-a9b2-76877a99b20a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:57.692777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-81e18316-f01c-4d4d-a9b2-76877a99b20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:57.763996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 121/236] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:04:57.792567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fe975-686c-4866-9cde-6ec5e5f71492 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:57.792885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fe975-686c-4866-9cde-6ec5e5f71492 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:57.793110 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fe975-686c-4866-9cde-6ec5e5f71492 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:57.794673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 116/237] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:57.799867 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01b737-3c90-43f8-b404-3d274431c246 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:57.800221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01b737-3c90-43f8-b404-3d274431c246 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:57.800501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01b737-3c90-43f8-b404-3d274431c246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:57.822313 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ad01b737-3c90-43f8-b404-3d274431c246 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:57.824061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ad01b737-3c90-43f8-b404-3d274431c246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:57.884673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:57.893148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 122/238] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:58.147085 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fe975-686c-4866-9cde-6ec5e5f71492 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:58.238252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01b737-3c90-43f8-b404-3d274431c246 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:58.273943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fe975-686c-4866-9cde-6ec5e5f71492 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:58.321607 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b977060-69a7-4da5-bc2e-9423efba54c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:58.322650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b977060-69a7-4da5-bc2e-9423efba54c5 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:58.323788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b977060-69a7-4da5-bc2e-9423efba54c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:58.329175 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b977060-69a7-4da5-bc2e-9423efba54c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:58.330739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b977060-69a7-4da5-bc2e-9423efba54c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:58.332427 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b977060-69a7-4da5-bc2e-9423efba54c5 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:58.336340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 117/239] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:04:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:58.339633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b977060-69a7-4da5-bc2e-9423efba54c5 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:58.433871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:58.451836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01b737-3c90-43f8-b404-3d274431c246 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:58.493160 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b977060-69a7-4da5-bc2e-9423efba54c5 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:58.535161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e76bc-8209-4ad0-b644-9fd047ec1841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:58.535846 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e76bc-8209-4ad0-b644-9fd047ec1841 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:58.536172 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e76bc-8209-4ad0-b644-9fd047ec1841 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:58.536450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e76bc-8209-4ad0-b644-9fd047ec1841 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:58.544060 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94b444-aa56-4008-a88d-ee66c36dbde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:58.545241 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94b444-aa56-4008-a88d-ee66c36dbde4 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:58.545919 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94b444-aa56-4008-a88d-ee66c36dbde4 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:58.546397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94b444-aa56-4008-a88d-ee66c36dbde4 None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:58.547625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94b444-aa56-4008-a88d-ee66c36dbde4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:58.549674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94b444-aa56-4008-a88d-ee66c36dbde4 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:58.550827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e76bc-8209-4ad0-b644-9fd047ec1841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:58.557237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-0e6e76bc-8209-4ad0-b644-9fd047ec1841 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:04:58.557237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:58.557237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:58.557237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:58.557237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:58.557237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:58.557237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:58.557237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:58.557237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:58.557237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:58.557237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:58.557237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:58.557237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:58.557237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:58.557237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:58.557237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:58.558685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:58.558685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:58.558685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:58.558685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:58.558685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:58.558685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:58.558685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:58.558685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:58.558685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:58.558685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:58.558685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:58.558685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:58.558685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:58.558685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:58.558685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:58.558685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:58.559504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:58.559504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:58.559504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:58.559504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:58.559504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:58.559504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:58.559504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:58.559504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:58.559504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:58.559504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:58.559504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:04:58.559504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:04:58.563322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 123/240] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:58.565673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588002e-12db-4cc4-abc8-51dfea9804fb None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:58.565959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588002e-12db-4cc4-abc8-51dfea9804fb None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:58.566232 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588002e-12db-4cc4-abc8-51dfea9804fb None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:58.568270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 124/241] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:58.571108 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588002e-12db-4cc4-abc8-51dfea9804fb None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:58.629344 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:58.639513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 118/242] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:04:58 2022] POST /identity/v3/users => generated 293 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:04:58.642288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcfc23-e595-4ba1-8d77-d6e88069d81e None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:58.642924 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcfc23-e595-4ba1-8d77-d6e88069d81e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:58.643442 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcfc23-e595-4ba1-8d77-d6e88069d81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:58.671494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-0cdcfc23-e595-4ba1-8d77-d6e88069d81e None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:04:58.674841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-0cdcfc23-e595-4ba1-8d77-d6e88069d81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:04:58.681824 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588002e-12db-4cc4-abc8-51dfea9804fb None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:58.719154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c5b79-cc61-4fab-aad0-65c35f8c331c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:58.719760 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c5b79-cc61-4fab-aad0-65c35f8c331c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:58.720040 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c5b79-cc61-4fab-aad0-65c35f8c331c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:58.720313 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c5b79-cc61-4fab-aad0-65c35f8c331c None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:58.721002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c5b79-cc61-4fab-aad0-65c35f8c331c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:58.721309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c5b79-cc61-4fab-aad0-65c35f8c331c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:58.723010 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c5b79-cc61-4fab-aad0-65c35f8c331c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:58.733236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 125/243] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:04:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:58.742695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c5b79-cc61-4fab-aad0-65c35f8c331c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:58.828794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 119/244] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:04:58.882247 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c5b79-cc61-4fab-aad0-65c35f8c331c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:58.899307 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:58.921281 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d3020-d00d-42f8-aaa5-ffff3531d206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:58.922281 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d3020-d00d-42f8-aaa5-ffff3531d206 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:58.922768 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d3020-d00d-42f8-aaa5-ffff3531d206 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:58.923309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d3020-d00d-42f8-aaa5-ffff3531d206 None admin] PATH_INFO: `/v3/users/055cff85dba4407baea3d2835f7d31ae` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:58.941562 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d3020-d00d-42f8-aaa5-ffff3531d206 None admin] RBAC: Authorizing `identity:get_user(user_id=055cff85dba4407baea3d2835f7d31ae)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:58.943392 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d3020-d00d-42f8-aaa5-ffff3531d206 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:58.945633 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 126/245] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:58 2022] GET /identity/v3/users/055cff85dba4407baea3d2835f7d31ae => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:58.950937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdcfc23-e595-4ba1-8d77-d6e88069d81e None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:59.006535 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:59.104904 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdcfc23-e595-4ba1-8d77-d6e88069d81e None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:59.161568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a4efb-06f9-4f94-9b9f-d0066edbeef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:59.162188 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a4efb-06f9-4f94-9b9f-d0066edbeef3 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:59.162511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a4efb-06f9-4f94-9b9f-d0066edbeef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:59.165796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a4efb-06f9-4f94-9b9f-d0066edbeef3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:59.167340 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d3020-d00d-42f8-aaa5-ffff3531d206 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:59.250497 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a4efb-06f9-4f94-9b9f-d0066edbeef3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:59.253462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a4efb-06f9-4f94-9b9f-d0066edbeef3 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:59.293845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d3020-d00d-42f8-aaa5-ffff3531d206 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:59.298807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:59.351000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941eacca-a625-4878-a8d3-4bf30906102c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:59.352097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941eacca-a625-4878-a8d3-4bf30906102c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:59.352857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941eacca-a625-4878-a8d3-4bf30906102c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:59.354279 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941eacca-a625-4878-a8d3-4bf30906102c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:59.370492 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941eacca-a625-4878-a8d3-4bf30906102c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:04:59.377466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-941eacca-a625-4878-a8d3-4bf30906102c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:04:59.377466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:04:59.377466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:04:59.377466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:04:59.377466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:04:59.377466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:04:59.377466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:04:59.377466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:04:59.377466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:04:59.377466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:04:59.377466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:04:59.377466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:04:59.377466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:04:59.377466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:04:59.377466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:04:59.377466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:04:59.378388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:04:59.378388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:04:59.378388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:04:59.378388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:04:59.378388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:04:59.378388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:04:59.378388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:04:59.378388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:04:59.378388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:04:59.378388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:04:59.378388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:04:59.378388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:04:59.378388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:04:59.378388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:04:59.378388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:04:59.378388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:04:59.379484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:04:59.379484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:04:59.379484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:04:59.379484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:04:59.379484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:04:59.379484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:04:59.379484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:04:59.379484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:04:59.379484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:04:59.379484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:04:59.379484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:04:59.379484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:04:59.381183 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 127/246] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:04:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:04:59.388274 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941eacca-a625-4878-a8d3-4bf30906102c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:59.407600 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 120/247] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:04:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:04:59.414145 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a4efb-06f9-4f94-9b9f-d0066edbeef3 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:59.439069 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:59.575807 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941eacca-a625-4878-a8d3-4bf30906102c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:59.602006 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a4efb-06f9-4f94-9b9f-d0066edbeef3 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:59.652415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb77d4-2b13-4b65-a422-9617298012f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:59.653051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb77d4-2b13-4b65-a422-9617298012f1 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:59.653330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb77d4-2b13-4b65-a422-9617298012f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:59.653605 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb77d4-2b13-4b65-a422-9617298012f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:59.654280 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb77d4-2b13-4b65-a422-9617298012f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:04:59.654583 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb77d4-2b13-4b65-a422-9617298012f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:59.656023 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb77d4-2b13-4b65-a422-9617298012f1 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:59.669163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 128/248] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:04:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:59.674475 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb77d4-2b13-4b65-a422-9617298012f1 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:59.675595 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4e8c0-9f89-4d49-bf76-717e2be4f5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:59.676793 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4e8c0-9f89-4d49-bf76-717e2be4f5bc None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:59.677282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4e8c0-9f89-4d49-bf76-717e2be4f5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:59.677747 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4e8c0-9f89-4d49-bf76-717e2be4f5bc None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:59.683906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4e8c0-9f89-4d49-bf76-717e2be4f5bc None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:59.685618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4e8c0-9f89-4d49-bf76-717e2be4f5bc None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:59.687543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 121/249] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:59 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:59.693407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4e8c0-9f89-4d49-bf76-717e2be4f5bc None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:59.700265 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:59.875880 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:04:59.880794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb77d4-2b13-4b65-a422-9617298012f1 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:59.941031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4e8c0-9f89-4d49-bf76-717e2be4f5bc None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:04:59.955737 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9585de-77f2-4e63-9487-9ffab2d84bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:59.956741 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9585de-77f2-4e63-9487-9ffab2d84bff None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:59.957272 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9585de-77f2-4e63-9487-9ffab2d84bff None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:59.957740 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9585de-77f2-4e63-9487-9ffab2d84bff None admin] PATH_INFO: `/v3/users/22b962825ca243418d673141782bec06` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:59.959854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9585de-77f2-4e63-9487-9ffab2d84bff None admin] RBAC: Authorizing `identity:get_user(user_id=22b962825ca243418d673141782bec06)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:04:59.961521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9585de-77f2-4e63-9487-9ffab2d84bff None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:04:59.963558 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 129/250] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:04:59 2022] GET /identity/v3/users/22b962825ca243418d673141782bec06 => generated 297 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:04:59.966609 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caa6dc-c6d9-4ed3-8310-e7895f116fed None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:59.967958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caa6dc-c6d9-4ed3-8310-e7895f116fed None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:59.969302 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caa6dc-c6d9-4ed3-8310-e7895f116fed None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:04:59.971088 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 130/251] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:04:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:04:59.975440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27caa6dc-c6d9-4ed3-8310-e7895f116fed None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:04:59.997216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435bffb-9424-4f85-9807-46a7ecb9fecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:04:59.998764 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435bffb-9424-4f85-9807-46a7ecb9fecb None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:04:59.999276 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435bffb-9424-4f85-9807-46a7ecb9fecb None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:04:59.999997 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435bffb-9424-4f85-9807-46a7ecb9fecb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:00.001203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435bffb-9424-4f85-9807-46a7ecb9fecb None admin] RBAC: Adding query filter params (role.id=05303606367c4086bb8e3c0c20d45b56, scope.project.id=1453ab88d2354019a189c0e0883fd8b1, user.id=055cff85dba4407baea3d2835f7d31ae) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:00.002213 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435bffb-9424-4f85-9807-46a7ecb9fecb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:00.004825 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435bffb-9424-4f85-9807-46a7ecb9fecb None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:00.023067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 122/252] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:04:59 2022] GET /identity/v3/role_assignments?user.id=055cff85dba4407baea3d2835f7d31ae&scope.project.id=1453ab88d2354019a189c0e0883fd8b1&role.id=05303606367c4086bb8e3c0c20d45b56 => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:00.023067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb7a2c-7197-457c-82f8-65043ecec2eb None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:00.023067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb7a2c-7197-457c-82f8-65043ecec2eb None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:00.023067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb7a2c-7197-457c-82f8-65043ecec2eb None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:00.023067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 123/253] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:00.023067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6457a5-77ee-4632-bdd3-1092e38da6d3 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:00.023657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6457a5-77ee-4632-bdd3-1092e38da6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:00.023657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6457a5-77ee-4632-bdd3-1092e38da6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:00.065948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-5c6457a5-77ee-4632-bdd3-1092e38da6d3 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:00.068040 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5c6457a5-77ee-4632-bdd3-1092e38da6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:00.180142 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:00.186900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 124/254] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:00.195197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30866996-37f4-4a2d-bef8-4d7d2566128d None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:00.196387 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30866996-37f4-4a2d-bef8-4d7d2566128d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:00.196889 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30866996-37f4-4a2d-bef8-4d7d2566128d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:00.227954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-30866996-37f4-4a2d-bef8-4d7d2566128d None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:00.230101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-30866996-37f4-4a2d-bef8-4d7d2566128d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:00.240904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27caa6dc-c6d9-4ed3-8310-e7895f116fed None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:00.319970 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc8f8e-c24f-42d5-b4d3-3a37e7da213a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:00.320989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 125/255] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:00.321496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc8f8e-c24f-42d5-b4d3-3a37e7da213a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:00.321961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc8f8e-c24f-42d5-b4d3-3a37e7da213a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:00.322487 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc8f8e-c24f-42d5-b4d3-3a37e7da213a None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:00.332069 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc8f8e-c24f-42d5-b4d3-3a37e7da213a None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:00.333832 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc8f8e-c24f-42d5-b4d3-3a37e7da213a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:00.336158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 131/256] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:04:59 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:00.341633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30866996-37f4-4a2d-bef8-4d7d2566128d None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:00.495090 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30866996-37f4-4a2d-bef8-4d7d2566128d None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:00.530210 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:00.537984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6541f-2b89-445b-97c3-21e645c93b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:00.538662 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6541f-2b89-445b-97c3-21e645c93b5f None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:00.538942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6541f-2b89-445b-97c3-21e645c93b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:00.539243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6541f-2b89-445b-97c3-21e645c93b5f None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1/users/22b962825ca243418d673141782bec06/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:00.541170 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6541f-2b89-445b-97c3-21e645c93b5f None admin] RBAC: Authorizing `identity:create_grant(project_id=1453ab88d2354019a189c0e0883fd8b1, user_id=22b962825ca243418d673141782bec06, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:00.543215 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6541f-2b89-445b-97c3-21e645c93b5f None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:00.589914 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc8f8e-c24f-42d5-b4d3-3a37e7da213a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:00.639196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:00.648941 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 126/257] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:05:00 2022] PUT /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1/users/22b962825ca243418d673141782bec06/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:05:00.721580 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc8f8e-c24f-42d5-b4d3-3a37e7da213a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:00.760882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f867b-47c3-4285-afd0-0453944747d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:00.761834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f867b-47c3-4285-afd0-0453944747d8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:00.762296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f867b-47c3-4285-afd0-0453944747d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:00.762764 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f867b-47c3-4285-afd0-0453944747d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:00.774814 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f867b-47c3-4285-afd0-0453944747d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:00.780772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-0c8f867b-47c3-4285-afd0-0453944747d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:05:00.780772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:00.780772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:00.780772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:00.780772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:00.780772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:00.780772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:00.780772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:00.780772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:00.780772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:00.780772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:00.780772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:00.780772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:00.780772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:00.780772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:00.780772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:00.781726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:00.781726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:00.781726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:00.781726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:00.781726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:00.781726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:00.781726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:00.781726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:00.781726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:00.781726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:00.781726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:00.781726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:00.781726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:00.781726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:00.781726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:00.781726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:00.782500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:00.782500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:00.782500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:00.782500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:00.782500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:00.782500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:00.782500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:00.782500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:00.782500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:00.782500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:00.782500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:05:00.782500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:00.784598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 132/258] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:05:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:00.792670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6541f-2b89-445b-97c3-21e645c93b5f None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:00.964668 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6541f-2b89-445b-97c3-21e645c93b5f None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:00.999200 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f5356-5274-49aa-9538-93ec1168819d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:01.000107 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f5356-5274-49aa-9538-93ec1168819d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:01.000550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f5356-5274-49aa-9538-93ec1168819d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:01.001008 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f5356-5274-49aa-9538-93ec1168819d None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:01.001806 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f5356-5274-49aa-9538-93ec1168819d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:01.002244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f5356-5274-49aa-9538-93ec1168819d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:01.004084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f5356-5274-49aa-9538-93ec1168819d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:01.013254 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 127/259] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:01.019105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f867b-47c3-4285-afd0-0453944747d8 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:01.039705 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f5356-5274-49aa-9538-93ec1168819d None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:01.143589 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:01.190334 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:01.199913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f867b-47c3-4285-afd0-0453944747d8 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:01.242592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203af9ee-9103-48bb-b765-8305c90fd651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:01.245503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203af9ee-9103-48bb-b765-8305c90fd651 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:01.246626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203af9ee-9103-48bb-b765-8305c90fd651 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:01.247051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203af9ee-9103-48bb-b765-8305c90fd651 None admin] PATH_INFO: `/v3/users/e9c5aabeb22c44c5a885cfbf39259d6b` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:01.248996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203af9ee-9103-48bb-b765-8305c90fd651 None admin] RBAC: Authorizing `identity:get_user(user_id=e9c5aabeb22c44c5a885cfbf39259d6b)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:01.250869 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203af9ee-9103-48bb-b765-8305c90fd651 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:01.252864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f5356-5274-49aa-9538-93ec1168819d None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:01.256580 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 133/260] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:05:01 2022] GET /identity/v3/users/e9c5aabeb22c44c5a885cfbf39259d6b => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:01.262459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203af9ee-9103-48bb-b765-8305c90fd651 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:01.334003 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f63546-9e4f-4f31-8c32-da7e0e1fbc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:01.334898 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f63546-9e4f-4f31-8c32-da7e0e1fbc60 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:01.335352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f63546-9e4f-4f31-8c32-da7e0e1fbc60 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:01.335768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f63546-9e4f-4f31-8c32-da7e0e1fbc60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:01.354734 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f63546-9e4f-4f31-8c32-da7e0e1fbc60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:01.366876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-31f63546-9e4f-4f31-8c32-da7e0e1fbc60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:05:01.366876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:01.366876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:01.366876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:01.366876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:01.366876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:01.366876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:01.366876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:01.366876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:01.366876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:01.366876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:01.366876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:01.366876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:01.366876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:01.366876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:01.366876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:01.368077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:01.368077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:01.368077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:01.368077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:01.368077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:01.368077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:01.368077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:01.368077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:01.368077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:01.368077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:01.368077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:01.368077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:01.368077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:01.368077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:01.368077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:01.368077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:01.369084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:01.369084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:01.369084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:01.369084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:01.369084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:01.369084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:01.369084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:01.369084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:01.369084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:01.369084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:01.369084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:05:01.369084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:01.374230 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 128/261] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:05:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:01.380683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f63546-9e4f-4f31-8c32-da7e0e1fbc60 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:01.469947 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203af9ee-9103-48bb-b765-8305c90fd651 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:01.542210 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe87fb6-d2bf-4a99-bb57-83e2a6fdc399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:01.543104 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe87fb6-d2bf-4a99-bb57-83e2a6fdc399 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:01.543583 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe87fb6-d2bf-4a99-bb57-83e2a6fdc399 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:01.545081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe87fb6-d2bf-4a99-bb57-83e2a6fdc399 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:01.563151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f63546-9e4f-4f31-8c32-da7e0e1fbc60 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:01.583708 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:01.597542 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:01.625249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464049d9-d08a-4553-aef5-c1faede556ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:01.626276 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464049d9-d08a-4553-aef5-c1faede556ee None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:01.626276 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464049d9-d08a-4553-aef5-c1faede556ee None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:01.626778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464049d9-d08a-4553-aef5-c1faede556ee None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:01.627145 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464049d9-d08a-4553-aef5-c1faede556ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:01.627471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464049d9-d08a-4553-aef5-c1faede556ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:01.629208 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464049d9-d08a-4553-aef5-c1faede556ee None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:01.640161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe87fb6-d2bf-4a99-bb57-83e2a6fdc399 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:01.641866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe87fb6-d2bf-4a99-bb57-83e2a6fdc399 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:01.648005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 129/262] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:01.653392 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464049d9-d08a-4553-aef5-c1faede556ee None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:01.757349 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 134/263] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 11:05:01.764795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe87fb6-d2bf-4a99-bb57-83e2a6fdc399 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:01.843229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464049d9-d08a-4553-aef5-c1faede556ee None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:01.845346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:01.891714 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe87fb6-d2bf-4a99-bb57-83e2a6fdc399 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:01.915691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9849b-cad3-4880-9199-9b25a2f6d4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:01.916354 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9849b-cad3-4880-9199-9b25a2f6d4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:01.917170 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9849b-cad3-4880-9199-9b25a2f6d4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:01.917442 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9849b-cad3-4880-9199-9b25a2f6d4b4 None admin] PATH_INFO: `/v3/users/e9c5aabeb22c44c5a885cfbf39259d6b` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:01.918946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9849b-cad3-4880-9199-9b25a2f6d4b4 None admin] RBAC: Authorizing `identity:get_user(user_id=e9c5aabeb22c44c5a885cfbf39259d6b)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:01.920487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9849b-cad3-4880-9199-9b25a2f6d4b4 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:01.922199 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 130/264] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:05:01 2022] GET /identity/v3/users/e9c5aabeb22c44c5a885cfbf39259d6b => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:01.930056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9849b-cad3-4880-9199-9b25a2f6d4b4 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:01.958969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f153b63-5346-4cb1-b613-7d159ff1f6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:01.959699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153b63-5346-4cb1-b613-7d159ff1f6ce None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:01.959984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153b63-5346-4cb1-b613-7d159ff1f6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:01.960262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153b63-5346-4cb1-b613-7d159ff1f6ce None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:01.961401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f153b63-5346-4cb1-b613-7d159ff1f6ce None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:01.963088 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f153b63-5346-4cb1-b613-7d159ff1f6ce None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:01.965157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 135/265] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:01 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:01.970947 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f153b63-5346-4cb1-b613-7d159ff1f6ce None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:02.013695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:02.140750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9849b-cad3-4880-9199-9b25a2f6d4b4 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:02.154056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f153b63-5346-4cb1-b613-7d159ff1f6ce None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:02.213080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:02.213338 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26312f-44c2-4800-9798-f7273ef13302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:02.214351 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26312f-44c2-4800-9798-f7273ef13302 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:02.214617 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26312f-44c2-4800-9798-f7273ef13302 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:02.214888 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26312f-44c2-4800-9798-f7273ef13302 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:02.220521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d4bf2-a8a1-4668-94ee-6125d0c5ed56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:02.221144 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d4bf2-a8a1-4668-94ee-6125d0c5ed56 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:02.221429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d4bf2-a8a1-4668-94ee-6125d0c5ed56 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:02.221700 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d4bf2-a8a1-4668-94ee-6125d0c5ed56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:02.223219 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d4bf2-a8a1-4668-94ee-6125d0c5ed56 None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=1453ab88d2354019a189c0e0883fd8b1, user.id=e9c5aabeb22c44c5a885cfbf39259d6b) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:02.223531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d4bf2-a8a1-4668-94ee-6125d0c5ed56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:02.224987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d4bf2-a8a1-4668-94ee-6125d0c5ed56 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:02.239328 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 136/266] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:05:01 2022] GET /identity/v3/role_assignments?user.id=e9c5aabeb22c44c5a885cfbf39259d6b&scope.project.id=1453ab88d2354019a189c0e0883fd8b1&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:02.242203 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e554150-436e-4dd7-945d-1d67306ab6ff None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:02.242514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e554150-436e-4dd7-945d-1d67306ab6ff None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:02.242789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e554150-436e-4dd7-945d-1d67306ab6ff None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:02.245128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 137/267] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:02.251198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb02a4-5991-429a-ad36-587310062746 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:02.251488 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb02a4-5991-429a-ad36-587310062746 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:02.251758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb02a4-5991-429a-ad36-587310062746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:02.294227 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c9cb02a4-5991-429a-ad36-587310062746 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:02.295479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c9cb02a4-5991-429a-ad36-587310062746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:02.358865 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26312f-44c2-4800-9798-f7273ef13302 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:02.360399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26312f-44c2-4800-9798-f7273ef13302 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:02.408725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:02.466691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 138/268] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:02.468013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3d4d2-f9c5-41b4-8ef0-96eefcbfc0d0 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:02.468553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3d4d2-f9c5-41b4-8ef0-96eefcbfc0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:02.468832 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3d4d2-f9c5-41b4-8ef0-96eefcbfc0d0 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:02.470288 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 139/269] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:02.479461 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e5e2c-602a-4d28-9dfc-f104d4253d4f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:02.479879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e5e2c-602a-4d28-9dfc-f104d4253d4f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:02.480247 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e5e2c-602a-4d28-9dfc-f104d4253d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:02.506535 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 131/270] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:02.515095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26312f-44c2-4800-9798-f7273ef13302 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:02.517478 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-062e5e2c-602a-4d28-9dfc-f104d4253d4f None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:02.519919 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-062e5e2c-602a-4d28-9dfc-f104d4253d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:02.662724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 140/271] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:02.686420 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26312f-44c2-4800-9798-f7273ef13302 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:02.722496 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79e13e-9b88-464d-a7d7-dfc3e0755e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:02.723138 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79e13e-9b88-464d-a7d7-dfc3e0755e2d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:02.723444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79e13e-9b88-464d-a7d7-dfc3e0755e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:02.723715 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79e13e-9b88-464d-a7d7-dfc3e0755e2d None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:02.724864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79e13e-9b88-464d-a7d7-dfc3e0755e2d None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:02.726352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79e13e-9b88-464d-a7d7-dfc3e0755e2d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:02.728191 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 132/272] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:02 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:02.734502 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e5e2c-602a-4d28-9dfc-f104d4253d4f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:02.795906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:02.812037 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79e13e-9b88-464d-a7d7-dfc3e0755e2d None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:02.831576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:02.885083 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e5e2c-602a-4d28-9dfc-f104d4253d4f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:02.961180 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba58cec-7d9f-4ff8-a273-0ed1d8ffe20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:02.961762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba58cec-7d9f-4ff8-a273-0ed1d8ffe20c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:02.962048 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba58cec-7d9f-4ff8-a273-0ed1d8ffe20c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:02.962317 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba58cec-7d9f-4ff8-a273-0ed1d8ffe20c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:02.963154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba58cec-7d9f-4ff8-a273-0ed1d8ffe20c None admin] RBAC: Adding query filter params (role.id=b7eccd60e28d4b39b54879dad8230e0d, scope.project.id=1453ab88d2354019a189c0e0883fd8b1, user.id=e9c5aabeb22c44c5a885cfbf39259d6b) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:02.963591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba58cec-7d9f-4ff8-a273-0ed1d8ffe20c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:02.964965 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba58cec-7d9f-4ff8-a273-0ed1d8ffe20c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:02.978604 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 141/273] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:05:02 2022] GET /identity/v3/role_assignments?user.id=e9c5aabeb22c44c5a885cfbf39259d6b&scope.project.id=1453ab88d2354019a189c0e0883fd8b1&role.id=b7eccd60e28d4b39b54879dad8230e0d => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:02.996275 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79e13e-9b88-464d-a7d7-dfc3e0755e2d None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:03.051996 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fb318-12bb-4eec-b1da-752b191a1268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:03.052676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fb318-12bb-4eec-b1da-752b191a1268 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:03.052950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fb318-12bb-4eec-b1da-752b191a1268 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:03.053215 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fb318-12bb-4eec-b1da-752b191a1268 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:03.054437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46fb318-12bb-4eec-b1da-752b191a1268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:03.056079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46fb318-12bb-4eec-b1da-752b191a1268 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:03.056826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba58cec-7d9f-4ff8-a273-0ed1d8ffe20c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:03.059266 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 133/274] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:03.068249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fb318-12bb-4eec-b1da-752b191a1268 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:03.211657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:03.247632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba58cec-7d9f-4ff8-a273-0ed1d8ffe20c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:03.263350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fb318-12bb-4eec-b1da-752b191a1268 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:03.283577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:03.314274 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78505543-a714-453c-98d2-a9aca4b7073a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:03.314924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78505543-a714-453c-98d2-a9aca4b7073a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:03.315238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78505543-a714-453c-98d2-a9aca4b7073a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:03.315516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78505543-a714-453c-98d2-a9aca4b7073a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:03.330242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78505543-a714-453c-98d2-a9aca4b7073a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:03.331347 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2da6c9-860c-443e-a307-250a2d0a834e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:03.332218 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2da6c9-860c-443e-a307-250a2d0a834e None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:03.332701 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2da6c9-860c-443e-a307-250a2d0a834e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:03.333202 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2da6c9-860c-443e-a307-250a2d0a834e None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:03.334061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2da6c9-860c-443e-a307-250a2d0a834e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:03.335670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-78505543-a714-453c-98d2-a9aca4b7073a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:03.335670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:03.335670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:03.335670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:03.335670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:03.335670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:03.335670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:03.335670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:03.335670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:03.335670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:03.335670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:03.335670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:03.335670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:03.335670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:03.335670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:03.335670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:03.336648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:03.336648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:03.336648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:03.336648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:03.336648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:03.336648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:03.336648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:03.336648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:03.336648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:03.336648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:03.336648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:03.336648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:03.336648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:03.336648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:03.336648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:03.336648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:03.337430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:03.337430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:03.337430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:03.337430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:03.337430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:03.337430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:03.337430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:03.337430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:03.337430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:03.337430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:03.337430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:03.337430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:03.337430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 142/275] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:03.338624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2da6c9-860c-443e-a307-250a2d0a834e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:03.344426 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78505543-a714-453c-98d2-a9aca4b7073a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:03.462760 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78505543-a714-453c-98d2-a9aca4b7073a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:03.464251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 134/276] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:05:03 2022] POST /identity/v3/users => generated 293 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:05:03.465315 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881ff0d-03f2-430b-9a41-56980b07702a None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:03.465602 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881ff0d-03f2-430b-9a41-56980b07702a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:03.465925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881ff0d-03f2-430b-9a41-56980b07702a None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:03.471455 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 135/277] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:03.473850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8635aba-92a9-41a1-afc6-a8cb045cbd16 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:03.474143 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8635aba-92a9-41a1-afc6-a8cb045cbd16 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:03.474418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8635aba-92a9-41a1-afc6-a8cb045cbd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:03.499227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-c8635aba-92a9-41a1-afc6-a8cb045cbd16 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:03.500851 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c8635aba-92a9-41a1-afc6-a8cb045cbd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:03.511087 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3169c1b-ea20-4d77-8503-435eeef19d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:03.512609 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3169c1b-ea20-4d77-8503-435eeef19d08 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:03.513211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3169c1b-ea20-4d77-8503-435eeef19d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:03.513777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3169c1b-ea20-4d77-8503-435eeef19d08 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:03.514656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3169c1b-ea20-4d77-8503-435eeef19d08 None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:03.515282 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3169c1b-ea20-4d77-8503-435eeef19d08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:03.516974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3169c1b-ea20-4d77-8503-435eeef19d08 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:03.530436 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 143/278] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:05:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:03.536736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3169c1b-ea20-4d77-8503-435eeef19d08 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:03.593155 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:03.599827 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 136/279] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:03.672168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3169c1b-ea20-4d77-8503-435eeef19d08 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:03.709860 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8a4fb-cc40-42b5-8ab2-7e4f3ef4ac7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:03.710519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8a4fb-cc40-42b5-8ab2-7e4f3ef4ac7a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:03.710797 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8a4fb-cc40-42b5-8ab2-7e4f3ef4ac7a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:03.711068 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8a4fb-cc40-42b5-8ab2-7e4f3ef4ac7a None admin] PATH_INFO: `/v3/users/8ac302f871574494a2051e020256326a` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:03.727590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8a4fb-cc40-42b5-8ab2-7e4f3ef4ac7a None admin] RBAC: Authorizing `identity:get_user(user_id=8ac302f871574494a2051e020256326a)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:03.729072 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8a4fb-cc40-42b5-8ab2-7e4f3ef4ac7a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:03.731357 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 144/280] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:05:03 2022] GET /identity/v3/users/8ac302f871574494a2051e020256326a => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:03.738176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8635aba-92a9-41a1-afc6-a8cb045cbd16 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:03.766832 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb51cd-1f11-446d-9927-dd37e5f3e335 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:03.767434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb51cd-1f11-446d-9927-dd37e5f3e335 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:03.767861 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb51cd-1f11-446d-9927-dd37e5f3e335 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:03.769367 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 145/281] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:03.775545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449c654-82d4-41ed-89c3-1865820ddad4 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:03.776445 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449c654-82d4-41ed-89c3-1865820ddad4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:03.776722 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449c654-82d4-41ed-89c3-1865820ddad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:03.799327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0449c654-82d4-41ed-89c3-1865820ddad4 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:03.937644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0449c654-82d4-41ed-89c3-1865820ddad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:03.937800 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:03.937800 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8635aba-92a9-41a1-afc6-a8cb045cbd16 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:03.937984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 146/282] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:05:03.937984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449c654-82d4-41ed-89c3-1865820ddad4 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:03.938143 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc888b-c934-4126-8c15-cdec95aba709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:03.938143 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc888b-c934-4126-8c15-cdec95aba709 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:03.938143 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc888b-c934-4126-8c15-cdec95aba709 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:03.938143 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc888b-c934-4126-8c15-cdec95aba709 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:03.938469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc888b-c934-4126-8c15-cdec95aba709 None admin] RBAC: Adding query filter params (role.id=486614b0ab4b454b92e9ef0bc7ec61c1, user.id=8ac302f871574494a2051e020256326a, scope.system=all) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:03.938469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc888b-c934-4126-8c15-cdec95aba709 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:03.938469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc888b-c934-4126-8c15-cdec95aba709 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:03.938469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 137/283] 10.222.0.78 () {56 vars in 1308 bytes} [Mon Sep 19 11:05:03 2022] GET /identity/v3/role_assignments?user.id=8ac302f871574494a2051e020256326a&scope.system=all&role.id=486614b0ab4b454b92e9ef0bc7ec61c1 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:03.943012 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:04.042259 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449c654-82d4-41ed-89c3-1865820ddad4 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:04.095922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0637fb7-c8e5-4ec8-a72c-866029e303c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:04.096597 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0637fb7-c8e5-4ec8-a72c-866029e303c2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:04.096887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0637fb7-c8e5-4ec8-a72c-866029e303c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:04.097161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0637fb7-c8e5-4ec8-a72c-866029e303c2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:04.116675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0637fb7-c8e5-4ec8-a72c-866029e303c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:04.122420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-e0637fb7-c8e5-4ec8-a72c-866029e303c2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:05:04.122420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:04.122420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:04.122420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:04.122420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:04.122420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:04.122420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:04.122420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:04.122420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:04.122420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:04.122420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:04.122420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:04.122420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:04.122420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:04.122420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:04.122420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:04.123446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:04.123446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:04.123446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:04.123446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:04.123446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:04.123446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:04.123446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:04.123446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:04.123446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:04.123446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:04.123446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:04.123446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:04.123446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:04.123446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:04.123446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:04.123446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:04.124267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:04.124267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:04.124267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:04.124267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:04.124267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:04.124267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:04.124267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:04.124267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:04.124267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:04.124267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:04.124267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:05:04.124267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:04.126750 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 147/284] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:04.133079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc888b-c934-4126-8c15-cdec95aba709 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:04.196463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0637fb7-c8e5-4ec8-a72c-866029e303c2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:04.268517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc888b-c934-4126-8c15-cdec95aba709 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:04.308034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:04.326142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:04.328776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdb634-407b-4555-aa11-b1be4e73e313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:04.329414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdb634-407b-4555-aa11-b1be4e73e313 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:04.329695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdb634-407b-4555-aa11-b1be4e73e313 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:04.329967 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdb634-407b-4555-aa11-b1be4e73e313 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:04.333855 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdb634-407b-4555-aa11-b1be4e73e313 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:04.334172 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdb634-407b-4555-aa11-b1be4e73e313 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:04.335652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdb634-407b-4555-aa11-b1be4e73e313 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:04.355865 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 138/285] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:05:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:04.358795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0637fb7-c8e5-4ec8-a72c-866029e303c2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:04.361523 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdb634-407b-4555-aa11-b1be4e73e313 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:04.424182 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d276c-9a4f-4188-b6eb-f57b64b8b074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:04.424838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d276c-9a4f-4188-b6eb-f57b64b8b074 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:04.425121 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d276c-9a4f-4188-b6eb-f57b64b8b074 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:04.425392 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d276c-9a4f-4188-b6eb-f57b64b8b074 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:04.446792 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d276c-9a4f-4188-b6eb-f57b64b8b074 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:04.452398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-750d276c-9a4f-4188-b6eb-f57b64b8b074 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:04.452398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:04.452398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:04.452398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:04.452398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:04.452398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:04.452398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:04.452398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:04.452398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:04.452398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:04.452398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:04.452398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:04.452398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:04.452398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:04.452398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:04.452398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:04.453296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:04.453296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:04.453296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:04.453296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:04.453296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:04.453296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:04.453296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:04.453296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:04.453296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:04.453296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:04.453296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:04.453296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:04.453296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:04.453296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:04.453296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:04.453296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:04.454080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:04.454080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:04.454080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:04.454080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:04.454080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:04.454080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:04.454080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:04.454080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:04.454080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:04.454080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:04.454080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:04.454080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:04.454080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 148/286] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:04.461156 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d276c-9a4f-4188-b6eb-f57b64b8b074 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:04.579139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdb634-407b-4555-aa11-b1be4e73e313 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:04.625153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d276c-9a4f-4188-b6eb-f57b64b8b074 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:04.666110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c456455-6b53-477f-8234-2776eac49b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:04.667295 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c456455-6b53-477f-8234-2776eac49b5e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:04.667577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c456455-6b53-477f-8234-2776eac49b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:04.667846 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c456455-6b53-477f-8234-2776eac49b5e None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:04.668513 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c456455-6b53-477f-8234-2776eac49b5e None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:04.668816 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c456455-6b53-477f-8234-2776eac49b5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:04.670184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c456455-6b53-477f-8234-2776eac49b5e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:04.676317 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e2407-d4ab-485a-ad97-60e02119c2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:04.676967 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e2407-d4ab-485a-ad97-60e02119c2e0 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:04.677244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e2407-d4ab-485a-ad97-60e02119c2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:04.677513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e2407-d4ab-485a-ad97-60e02119c2e0 None admin] PATH_INFO: `/v3/users/055cff85dba4407baea3d2835f7d31ae` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:04.678710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e2407-d4ab-485a-ad97-60e02119c2e0 None admin] RBAC: Authorizing `identity:get_user(user_id=055cff85dba4407baea3d2835f7d31ae)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:04.680196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e2407-d4ab-485a-ad97-60e02119c2e0 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:04.686825 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 139/287] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:05:04 2022] GET /identity/v3/users/055cff85dba4407baea3d2835f7d31ae => generated 297 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:04.688213 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e2407-d4ab-485a-ad97-60e02119c2e0 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:04.689862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 149/288] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:05:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:04.696364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c456455-6b53-477f-8234-2776eac49b5e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:04.714938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:04.796477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:04.816226 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c456455-6b53-477f-8234-2776eac49b5e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:04.857018 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe1854-efdc-4433-aba5-7effa764a02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:04.860969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe1854-efdc-4433-aba5-7effa764a02d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:04.861217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe1854-efdc-4433-aba5-7effa764a02d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:04.861490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe1854-efdc-4433-aba5-7effa764a02d None admin] PATH_INFO: `/v3/users/22b962825ca243418d673141782bec06` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:04.863677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe1854-efdc-4433-aba5-7effa764a02d None admin] RBAC: Authorizing `identity:get_user(user_id=22b962825ca243418d673141782bec06)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:04.865122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe1854-efdc-4433-aba5-7effa764a02d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:04.866890 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 150/289] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:05:04 2022] GET /identity/v3/users/22b962825ca243418d673141782bec06 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:04.872993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe1854-efdc-4433-aba5-7effa764a02d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:04.882777 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e2407-d4ab-485a-ad97-60e02119c2e0 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:04.949736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ebbb5-b6ee-4f5a-bb67-8dfe12d92ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:04.950400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ebbb5-b6ee-4f5a-bb67-8dfe12d92ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:04.950672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ebbb5-b6ee-4f5a-bb67-8dfe12d92ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:04.950938 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ebbb5-b6ee-4f5a-bb67-8dfe12d92ba3 None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:04.952903 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ebbb5-b6ee-4f5a-bb67-8dfe12d92ba3 None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:04.954406 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ebbb5-b6ee-4f5a-bb67-8dfe12d92ba3 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:04.956520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 140/290] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:04 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:04.962578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ebbb5-b6ee-4f5a-bb67-8dfe12d92ba3 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:04.997663 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:05.034590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe1854-efdc-4433-aba5-7effa764a02d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:05.101403 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69fe70-e4a5-4310-b84f-31e4426a4590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:05.104733 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69fe70-e4a5-4310-b84f-31e4426a4590 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:05.105163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69fe70-e4a5-4310-b84f-31e4426a4590 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:05.105778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69fe70-e4a5-4310-b84f-31e4426a4590 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:05.173312 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ebbb5-b6ee-4f5a-bb67-8dfe12d92ba3 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:05.217634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:05.229888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69fe70-e4a5-4310-b84f-31e4426a4590 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:05.231429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69fe70-e4a5-4310-b84f-31e4426a4590 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:05.244825 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9500f3-295e-4905-9aa2-3d7c2834e59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:05.245928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9500f3-295e-4905-9aa2-3d7c2834e59e None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:05.246419 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9500f3-295e-4905-9aa2-3d7c2834e59e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:05.246926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9500f3-295e-4905-9aa2-3d7c2834e59e None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1/users/055cff85dba4407baea3d2835f7d31ae/roles/05303606367c4086bb8e3c0c20d45b56` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:05.256013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9500f3-295e-4905-9aa2-3d7c2834e59e None admin] RBAC: Authorizing `identity:create_grant(project_id=1453ab88d2354019a189c0e0883fd8b1, user_id=055cff85dba4407baea3d2835f7d31ae, role_id=05303606367c4086bb8e3c0c20d45b56)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:05.257488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9500f3-295e-4905-9aa2-3d7c2834e59e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:05.317457 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:05.345323 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 151/291] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:05.345530 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 141/292] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:05:04 2022] PUT /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1/users/055cff85dba4407baea3d2835f7d31ae/roles/05303606367c4086bb8e3c0c20d45b56 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:05:05.353155 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9500f3-295e-4905-9aa2-3d7c2834e59e None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:05.531370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9500f3-295e-4905-9aa2-3d7c2834e59e None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:05.574758 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea1a57-afc1-4b1c-86a7-359cb191e5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:05.575439 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea1a57-afc1-4b1c-86a7-359cb191e5cd None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:05.575721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea1a57-afc1-4b1c-86a7-359cb191e5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:05.575994 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea1a57-afc1-4b1c-86a7-359cb191e5cd None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:05.577793 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea1a57-afc1-4b1c-86a7-359cb191e5cd None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:05.579286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea1a57-afc1-4b1c-86a7-359cb191e5cd None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:05.581098 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 142/293] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:05 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:05.587157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69fe70-e4a5-4310-b84f-31e4426a4590 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:05.692802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69fe70-e4a5-4310-b84f-31e4426a4590 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:05.728457 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983055f7-9274-4e59-9e96-7864f2e4266c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:05.729481 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983055f7-9274-4e59-9e96-7864f2e4266c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:05.729958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983055f7-9274-4e59-9e96-7864f2e4266c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:05.730431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983055f7-9274-4e59-9e96-7864f2e4266c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:05.731382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983055f7-9274-4e59-9e96-7864f2e4266c None admin] RBAC: Adding query filter params (role.id=486614b0ab4b454b92e9ef0bc7ec61c1, scope.project.id=1453ab88d2354019a189c0e0883fd8b1, user.id=22b962825ca243418d673141782bec06) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:05.731906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983055f7-9274-4e59-9e96-7864f2e4266c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:05.733593 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983055f7-9274-4e59-9e96-7864f2e4266c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:05.740215 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 152/294] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:05:05 2022] GET /identity/v3/role_assignments?user.id=22b962825ca243418d673141782bec06&scope.project.id=1453ab88d2354019a189c0e0883fd8b1&role.id=486614b0ab4b454b92e9ef0bc7ec61c1 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:06.145584 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d28be-a040-408f-acfe-424a97c8aac5 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:06.145950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d28be-a040-408f-acfe-424a97c8aac5 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:06.146140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d28be-a040-408f-acfe-424a97c8aac5 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:06.147673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 143/295] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:06.153582 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d555a-a1b4-4edc-8ade-8113153ce164 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:06.153925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d555a-a1b4-4edc-8ade-8113153ce164 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:06.154205 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d555a-a1b4-4edc-8ade-8113153ce164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:06.178904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0c3d555a-a1b4-4edc-8ade-8113153ce164 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:06.180191 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0c3d555a-a1b4-4edc-8ade-8113153ce164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:06.226975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:06.265372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 153/296] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:06.439405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78823315-278c-4a92-925a-773605937374 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:06.439906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78823315-278c-4a92-925a-773605937374 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:06.439906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78823315-278c-4a92-925a-773605937374 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:06.441467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 144/297] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:06.446712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4203f-e464-4e6a-a2b9-d07ff86b073c None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:06.447007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4203f-e464-4e6a-a2b9-d07ff86b073c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:06.447309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4203f-e464-4e6a-a2b9-d07ff86b073c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:06.466128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d1c4203f-e464-4e6a-a2b9-d07ff86b073c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:06.467361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d1c4203f-e464-4e6a-a2b9-d07ff86b073c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:06.545967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 154/298] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:06.552292 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78823315-278c-4a92-925a-773605937374 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:06.610133 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:06.643499 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e504c74-7c4e-4dd0-b47b-188b7a2e9b33 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:06.644137 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e504c74-7c4e-4dd0-b47b-188b7a2e9b33 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:06.644788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e504c74-7c4e-4dd0-b47b-188b7a2e9b33 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:06.646320 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 155/299] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:06.652150 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4011610-3fe6-4915-b3b9-0fc3c2f136ba None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:06.652432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4011610-3fe6-4915-b3b9-0fc3c2f136ba None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:06.652705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4011610-3fe6-4915-b3b9-0fc3c2f136ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:06.667862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78823315-278c-4a92-925a-773605937374 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:06.672435 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c4011610-3fe6-4915-b3b9-0fc3c2f136ba None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:06.673622 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c4011610-3fe6-4915-b3b9-0fc3c2f136ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:06.705110 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d52fce0-5ff4-4266-ac0f-6f1f74eb0f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:06.706034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52fce0-5ff4-4266-ac0f-6f1f74eb0f2f None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:06.706493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52fce0-5ff4-4266-ac0f-6f1f74eb0f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:06.706939 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52fce0-5ff4-4266-ac0f-6f1f74eb0f2f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:06.718772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d52fce0-5ff4-4266-ac0f-6f1f74eb0f2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:06.724074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-5d52fce0-5ff4-4266-ac0f-6f1f74eb0f2f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:05:06.724074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:06.724074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:06.724074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:06.724074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:06.724074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:06.724074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:06.724074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:06.724074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:06.724074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:06.724074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:06.724074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:06.724074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:06.724074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:06.724074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:06.724074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:06.725032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:06.725032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:06.725032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:06.725032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:06.725032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:06.725032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:06.725032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:06.725032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:06.725032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:06.725032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:06.725032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:06.725032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:06.725032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:06.725032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:06.725032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:06.725032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:06.725833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:06.725833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:06.725833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:06.725833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:06.725833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:06.725833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:06.725833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:06.725833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:06.725833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:06.725833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:06.725833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:05:06.725833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:06.728186 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 145/300] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:06.733946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d52fce0-5ff4-4266-ac0f-6f1f74eb0f2f None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:06.759264 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 156/301] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:05:06.837836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4011610-3fe6-4915-b3b9-0fc3c2f136ba None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:06.844521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d52fce0-5ff4-4266-ac0f-6f1f74eb0f2f None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:06.862432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:06.872439 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:06.894618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224f031-549a-4de2-9283-d2d1d550aae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:06.895246 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224f031-549a-4de2-9283-d2d1d550aae0 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:06.895526 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224f031-549a-4de2-9283-d2d1d550aae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:06.895797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224f031-549a-4de2-9283-d2d1d550aae0 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:06.896516 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224f031-549a-4de2-9283-d2d1d550aae0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:06.896814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224f031-549a-4de2-9283-d2d1d550aae0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:06.898204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224f031-549a-4de2-9283-d2d1d550aae0 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:06.907447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 146/302] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:05:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:06.912693 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224f031-549a-4de2-9283-d2d1d550aae0 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:06.964614 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4011610-3fe6-4915-b3b9-0fc3c2f136ba None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:07.012722 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfda808-36d6-4d8c-b0e1-c6faaedd796c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:07.014568 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfda808-36d6-4d8c-b0e1-c6faaedd796c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:07.015510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfda808-36d6-4d8c-b0e1-c6faaedd796c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:07.016221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfda808-36d6-4d8c-b0e1-c6faaedd796c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:07.031627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfda808-36d6-4d8c-b0e1-c6faaedd796c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:07.037110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-bcfda808-36d6-4d8c-b0e1-c6faaedd796c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:07.037110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:07.037110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:07.037110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:07.037110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:07.037110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:07.037110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:07.037110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:07.037110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:07.037110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:07.037110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:07.037110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:07.037110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:07.037110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:07.037110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:07.037110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:07.037987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:07.037987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:07.037987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:07.037987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:07.037987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:07.037987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:07.037987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:07.037987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:07.037987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:07.037987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:07.037987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:07.037987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:07.037987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:07.037987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:07.037987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:07.037987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:07.038758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:07.038758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:07.038758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:07.038758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:07.038758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:07.038758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:07.038758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:07.038758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:07.038758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:07.038758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:07.038758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:07.038758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:07.039474 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224f031-549a-4de2-9283-d2d1d550aae0 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:07.039548 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 157/303] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:07.040435 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfda808-36d6-4d8c-b0e1-c6faaedd796c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:07.078309 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ed4ec-d0f0-4a7c-9f0a-19f75ae8008b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:07.078947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ed4ec-d0f0-4a7c-9f0a-19f75ae8008b None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:07.079257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ed4ec-d0f0-4a7c-9f0a-19f75ae8008b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:07.079533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ed4ec-d0f0-4a7c-9f0a-19f75ae8008b None admin] PATH_INFO: `/v3/users/d116c243fbcd4fbb980a3727f6ba46e7` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:07.093239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ed4ec-d0f0-4a7c-9f0a-19f75ae8008b None admin] RBAC: Authorizing `identity:get_user(user_id=d116c243fbcd4fbb980a3727f6ba46e7)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:07.094710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ed4ec-d0f0-4a7c-9f0a-19f75ae8008b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:07.096403 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 147/304] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:05:06 2022] GET /identity/v3/users/d116c243fbcd4fbb980a3727f6ba46e7 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:07.097176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ed4ec-d0f0-4a7c-9f0a-19f75ae8008b None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:07.129894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:07.148317 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:07.162845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfda808-36d6-4d8c-b0e1-c6faaedd796c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:07.204925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4fc42-1193-4279-81f3-2215faf727ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:07.205756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4fc42-1193-4279-81f3-2215faf727ae None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:07.206347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4fc42-1193-4279-81f3-2215faf727ae None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:07.206638 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4fc42-1193-4279-81f3-2215faf727ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:07.208105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4fc42-1193-4279-81f3-2215faf727ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:07.208827 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ed4ec-d0f0-4a7c-9f0a-19f75ae8008b None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:07.209553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4fc42-1193-4279-81f3-2215faf727ae None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:07.212176 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 158/305] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:07.212967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4fc42-1193-4279-81f3-2215faf727ae None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:07.245656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc02f4f-1f58-49f3-b0d8-e6713e8a88b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:07.246521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc02f4f-1f58-49f3-b0d8-e6713e8a88b2 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:07.246805 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc02f4f-1f58-49f3-b0d8-e6713e8a88b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:07.247088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc02f4f-1f58-49f3-b0d8-e6713e8a88b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:07.247801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc02f4f-1f58-49f3-b0d8-e6713e8a88b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:07.248118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc02f4f-1f58-49f3-b0d8-e6713e8a88b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:07.249487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc02f4f-1f58-49f3-b0d8-e6713e8a88b2 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:07.259303 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 148/306] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:05:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:07.259303 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc02f4f-1f58-49f3-b0d8-e6713e8a88b2 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:07.325337 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4fc42-1193-4279-81f3-2215faf727ae None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:07.362236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:07.366168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c582e7-8d67-45a7-bb90-bfdc57de6c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:07.367110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c582e7-8d67-45a7-bb90-bfdc57de6c6a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:07.367704 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c582e7-8d67-45a7-bb90-bfdc57de6c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:07.368048 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c582e7-8d67-45a7-bb90-bfdc57de6c6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:07.368785 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c582e7-8d67-45a7-bb90-bfdc57de6c6a None admin] RBAC: Adding query filter params (role.id=05303606367c4086bb8e3c0c20d45b56, user.id=d116c243fbcd4fbb980a3727f6ba46e7, scope.system=all) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:07.369156 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c582e7-8d67-45a7-bb90-bfdc57de6c6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:07.369334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc02f4f-1f58-49f3-b0d8-e6713e8a88b2 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:07.370787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c582e7-8d67-45a7-bb90-bfdc57de6c6a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:07.376437 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 159/307] 10.222.0.78 () {56 vars in 1308 bytes} [Mon Sep 19 11:05:07 2022] GET /identity/v3/role_assignments?user.id=d116c243fbcd4fbb980a3727f6ba46e7&scope.system=all&role.id=05303606367c4086bb8e3c0c20d45b56 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:07.377314 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c582e7-8d67-45a7-bb90-bfdc57de6c6a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:07.420950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:07.440349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686ed87-36e8-4aa7-a1f5-d5f31c83d437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:07.443562 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686ed87-36e8-4aa7-a1f5-d5f31c83d437 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:07.443795 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686ed87-36e8-4aa7-a1f5-d5f31c83d437 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:07.445342 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686ed87-36e8-4aa7-a1f5-d5f31c83d437 None admin] PATH_INFO: `/v3/groups` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:07.446063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686ed87-36e8-4aa7-a1f5-d5f31c83d437 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:07.447687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686ed87-36e8-4aa7-a1f5-d5f31c83d437 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:07.521423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 149/308] 10.222.0.78 () {60 vars in 1134 bytes} [Mon Sep 19 11:05:07 2022] POST /identity/v3/groups => generated 229 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:05:07.537009 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c582e7-8d67-45a7-bb90-bfdc57de6c6a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:07.581025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e5efb-c866-4397-8741-1356b5d51ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:07.582201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e5efb-c866-4397-8741-1356b5d51ece None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:07.582683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e5efb-c866-4397-8741-1356b5d51ece None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:07.583227 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e5efb-c866-4397-8741-1356b5d51ece None admin] PATH_INFO: `/v3/users/8ac302f871574494a2051e020256326a` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:07.584761 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e5efb-c866-4397-8741-1356b5d51ece None admin] RBAC: Authorizing `identity:get_user(user_id=8ac302f871574494a2051e020256326a)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:07.586483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e5efb-c866-4397-8741-1356b5d51ece None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:07.589111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 160/309] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:05:07 2022] GET /identity/v3/users/8ac302f871574494a2051e020256326a => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:07.598079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686ed87-36e8-4aa7-a1f5-d5f31c83d437 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:07.712372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686ed87-36e8-4aa7-a1f5-d5f31c83d437 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:07.752624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4c7d3-35b1-4786-ae4b-64228c8147b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:07.753327 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4c7d3-35b1-4786-ae4b-64228c8147b7 None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:07.753928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4c7d3-35b1-4786-ae4b-64228c8147b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:07.754277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4c7d3-35b1-4786-ae4b-64228c8147b7 None admin] PATH_INFO: `/v3/system/users/8ac302f871574494a2051e020256326a/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:07.755993 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4c7d3-35b1-4786-ae4b-64228c8147b7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8ac302f871574494a2051e020256326a, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:07.757499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4c7d3-35b1-4786-ae4b-64228c8147b7 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:07.858634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 150/310] 10.222.0.78 () {58 vars in 1334 bytes} [Mon Sep 19 11:05:07 2022] PUT /identity/v3/system/users/8ac302f871574494a2051e020256326a/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:05:07.860960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751cc12-acc2-4a8a-b2d0-736f9bf90728 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:07.861403 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751cc12-acc2-4a8a-b2d0-736f9bf90728 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:07.861567 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751cc12-acc2-4a8a-b2d0-736f9bf90728 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:07.862884 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 161/311] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:07.868918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec2c09-6182-4224-8d3b-e1807bfa0849 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:07.873402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec2c09-6182-4224-8d3b-e1807bfa0849 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:07.873402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec2c09-6182-4224-8d3b-e1807bfa0849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:07.891491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-77ec2c09-6182-4224-8d3b-e1807bfa0849 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:07.892761 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-77ec2c09-6182-4224-8d3b-e1807bfa0849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:07.909683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:07.971394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 151/312] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:08.248920 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751cc12-acc2-4a8a-b2d0-736f9bf90728 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:08.353298 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:08.364603 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751cc12-acc2-4a8a-b2d0-736f9bf90728 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:08.400728 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad23a3-6525-4b0a-9c0f-1bb8823747f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:08.401424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad23a3-6525-4b0a-9c0f-1bb8823747f1 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:08.401710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad23a3-6525-4b0a-9c0f-1bb8823747f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:08.402045 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad23a3-6525-4b0a-9c0f-1bb8823747f1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:08.413141 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad23a3-6525-4b0a-9c0f-1bb8823747f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:08.418710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-27ad23a3-6525-4b0a-9c0f-1bb8823747f1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:05:08.418710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:08.418710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:08.418710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:08.418710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:08.418710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:08.418710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:08.418710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:08.418710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:08.418710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:08.418710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:08.418710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:08.418710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:08.418710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:08.418710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:08.418710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:08.419658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:08.419658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:08.419658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:08.419658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:08.419658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:08.419658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:08.419658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:08.419658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:08.419658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:08.419658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:08.419658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:08.419658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:08.419658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:08.419658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:08.419658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:08.419658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:08.420434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:08.420434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:08.420434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:08.420434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:08.420434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:08.420434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:08.420434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:08.420434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:08.420434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:08.420434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:08.420434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:05:08.420434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:08.420434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 162/313] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:08.425922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec2c09-6182-4224-8d3b-e1807bfa0849 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:08.528384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec2c09-6182-4224-8d3b-e1807bfa0849 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:08.565166 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2065d-6589-4a5b-b9c5-acf94a4402c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:08.566149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2065d-6589-4a5b-b9c5-acf94a4402c0 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:08.566624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2065d-6589-4a5b-b9c5-acf94a4402c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:08.567114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2065d-6589-4a5b-b9c5-acf94a4402c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:08.568003 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2065d-6589-4a5b-b9c5-acf94a4402c0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:08.568493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2065d-6589-4a5b-b9c5-acf94a4402c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:08.570157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2065d-6589-4a5b-b9c5-acf94a4402c0 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:08.579400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 152/314] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:05:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:08.584853 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad23a3-6525-4b0a-9c0f-1bb8823747f1 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:08.709183 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad23a3-6525-4b0a-9c0f-1bb8823747f1 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:08.749621 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771a53a-6576-4a50-9045-e0a6bd4023e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:08.750600 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771a53a-6576-4a50-9045-e0a6bd4023e1 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:08.751029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771a53a-6576-4a50-9045-e0a6bd4023e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:08.751471 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771a53a-6576-4a50-9045-e0a6bd4023e1 None admin] PATH_INFO: `/v3/users/055cff85dba4407baea3d2835f7d31ae` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:08.752773 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771a53a-6576-4a50-9045-e0a6bd4023e1 None admin] RBAC: Authorizing `identity:get_user(user_id=055cff85dba4407baea3d2835f7d31ae)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:08.754642 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771a53a-6576-4a50-9045-e0a6bd4023e1 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:08.756442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 163/315] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:05:08 2022] GET /identity/v3/users/055cff85dba4407baea3d2835f7d31ae => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:08.762325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2065d-6589-4a5b-b9c5-acf94a4402c0 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:08.779219 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:08.941567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2065d-6589-4a5b-b9c5-acf94a4402c0 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:08.984896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469abb4d-12d4-4375-8c31-b9a7c89a4544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:08.985757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469abb4d-12d4-4375-8c31-b9a7c89a4544 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:08.986181 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469abb4d-12d4-4375-8c31-b9a7c89a4544 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:08.986598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469abb4d-12d4-4375-8c31-b9a7c89a4544 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:09.131995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469abb4d-12d4-4375-8c31-b9a7c89a4544 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:09.133708 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469abb4d-12d4-4375-8c31-b9a7c89a4544 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:09.151333 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:09.291720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 153/316] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:09.297536 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771a53a-6576-4a50-9045-e0a6bd4023e1 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:09.319837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:09.410468 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771a53a-6576-4a50-9045-e0a6bd4023e1 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:09.449354 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd352cc1-c718-4e33-8b48-1f3b0b2e22fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:09.450008 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd352cc1-c718-4e33-8b48-1f3b0b2e22fd None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:09.450292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd352cc1-c718-4e33-8b48-1f3b0b2e22fd None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:09.450568 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd352cc1-c718-4e33-8b48-1f3b0b2e22fd None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:09.451778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd352cc1-c718-4e33-8b48-1f3b0b2e22fd None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:09.453221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd352cc1-c718-4e33-8b48-1f3b0b2e22fd None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:09.454936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 164/317] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:09 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:09.461037 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469abb4d-12d4-4375-8c31-b9a7c89a4544 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:09.568753 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469abb4d-12d4-4375-8c31-b9a7c89a4544 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:09.583582 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:09.611986 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f3666-84c5-4d04-9e51-3e804ed99ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:09.612622 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f3666-84c5-4d04-9e51-3e804ed99ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:09.612901 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f3666-84c5-4d04-9e51-3e804ed99ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:09.613173 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f3666-84c5-4d04-9e51-3e804ed99ec8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:09.613923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f3666-84c5-4d04-9e51-3e804ed99ec8 None admin] RBAC: Adding query filter params (role.id=05303606367c4086bb8e3c0c20d45b56, scope.project.id=1453ab88d2354019a189c0e0883fd8b1, user.id=055cff85dba4407baea3d2835f7d31ae) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:09.614226 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f3666-84c5-4d04-9e51-3e804ed99ec8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:09.615904 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f3666-84c5-4d04-9e51-3e804ed99ec8 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:09.623741 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 154/318] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:05:09 2022] GET /identity/v3/role_assignments?user.id=055cff85dba4407baea3d2835f7d31ae&scope.project.id=1453ab88d2354019a189c0e0883fd8b1&role.id=05303606367c4086bb8e3c0c20d45b56 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:09.719387 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d00a9c-427d-4cc4-b0e1-4cca5425e778 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:09.720132 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d00a9c-427d-4cc4-b0e1-4cca5425e778 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:09.720619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d00a9c-427d-4cc4-b0e1-4cca5425e778 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:09.722278 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 165/319] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:09.728270 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb83a82-9698-48a6-8db1-df0d1440187d None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:09.729482 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb83a82-9698-48a6-8db1-df0d1440187d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:09.729952 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb83a82-9698-48a6-8db1-df0d1440187d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:09.749836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-ddb83a82-9698-48a6-8db1-df0d1440187d None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:09.751556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ddb83a82-9698-48a6-8db1-df0d1440187d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:09.833085 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 155/320] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:09.886010 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebf73b-dbb3-4344-8852-e95325221147 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:09.886523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebf73b-dbb3-4344-8852-e95325221147 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:09.886935 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebf73b-dbb3-4344-8852-e95325221147 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:09.888482 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 166/321] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:09.896448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6615a7-4c7e-4a3d-91e5-75e17283cfb4 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:09.897138 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6615a7-4c7e-4a3d-91e5-75e17283cfb4 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:09.897751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6615a7-4c7e-4a3d-91e5-75e17283cfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:09.918687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-ca6615a7-4c7e-4a3d-91e5-75e17283cfb4 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:09.920034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ca6615a7-4c7e-4a3d-91e5-75e17283cfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:09.990187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 156/322] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:10.112652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebf73b-dbb3-4344-8852-e95325221147 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:10.210118 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:10.226356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebf73b-dbb3-4344-8852-e95325221147 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:10.258841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6615a7-4c7e-4a3d-91e5-75e17283cfb4 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:10.264871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445c452-545b-4bb1-a51f-e2c28552e669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:10.265516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445c452-545b-4bb1-a51f-e2c28552e669 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:10.265794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445c452-545b-4bb1-a51f-e2c28552e669 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:10.266066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445c452-545b-4bb1-a51f-e2c28552e669 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:10.279919 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445c452-545b-4bb1-a51f-e2c28552e669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:10.285308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-c445c452-545b-4bb1-a51f-e2c28552e669 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:05:10.285308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:10.285308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:10.285308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:10.285308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:10.285308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:10.285308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:10.285308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:10.285308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:10.285308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:10.285308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:10.285308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:10.285308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:10.285308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:10.285308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:10.285308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:10.286175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:10.286175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:10.286175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:10.286175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:10.286175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:10.286175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:10.286175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:10.286175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:10.286175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:10.286175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:10.286175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:10.286175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:10.286175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:10.286175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:10.286175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:10.286175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:10.288000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:10.288000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:10.288000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:10.288000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:10.288000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:10.288000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:10.288000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:10.288000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:10.288000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:10.288000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:10.288000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:05:10.288000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:10.288000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 167/323] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:10.288000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd4eb1-362c-4a00-9a94-2c74230d0f0b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:10.288710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd4eb1-362c-4a00-9a94-2c74230d0f0b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:10.288710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd4eb1-362c-4a00-9a94-2c74230d0f0b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:10.291075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 168/324] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:10.291962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd4eb1-362c-4a00-9a94-2c74230d0f0b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:10.297934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:10.367718 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6615a7-4c7e-4a3d-91e5-75e17283cfb4 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:10.398817 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd4eb1-362c-4a00-9a94-2c74230d0f0b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:10.403871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d09a61-3497-434f-847d-a65f7be6c0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:10.404511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09a61-3497-434f-847d-a65f7be6c0cf None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:10.404786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09a61-3497-434f-847d-a65f7be6c0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:10.405054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09a61-3497-434f-847d-a65f7be6c0cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:10.406147 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d09a61-3497-434f-847d-a65f7be6c0cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:10.407636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d09a61-3497-434f-847d-a65f7be6c0cf None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:10.409351 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 157/325] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:10.410425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b282143-7282-40fd-900e-91d1455ea0ed None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:10.410714 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b282143-7282-40fd-900e-91d1455ea0ed None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:10.410978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b282143-7282-40fd-900e-91d1455ea0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:10.429616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-0b282143-7282-40fd-900e-91d1455ea0ed None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:10.430819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-0b282143-7282-40fd-900e-91d1455ea0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:10.434175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05664e4b-e906-42e5-aa17-a4bb1c4b0917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:10.434800 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05664e4b-e906-42e5-aa17-a4bb1c4b0917 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:10.435084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05664e4b-e906-42e5-aa17-a4bb1c4b0917 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:10.435390 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05664e4b-e906-42e5-aa17-a4bb1c4b0917 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:10.436063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05664e4b-e906-42e5-aa17-a4bb1c4b0917 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:10.436371 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05664e4b-e906-42e5-aa17-a4bb1c4b0917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:10.437752 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05664e4b-e906-42e5-aa17-a4bb1c4b0917 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:10.446152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 169/326] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:05:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:10.446911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05664e4b-e906-42e5-aa17-a4bb1c4b0917 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:10.458237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:10.505617 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 158/327] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:10.506474 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b282143-7282-40fd-900e-91d1455ea0ed None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:10.541044 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:10.554621 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05664e4b-e906-42e5-aa17-a4bb1c4b0917 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:10.592139 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f724b4-eb5d-4a2c-ac87-0b0e53290a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:10.592758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f724b4-eb5d-4a2c-ac87-0b0e53290a80 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:10.593044 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f724b4-eb5d-4a2c-ac87-0b0e53290a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:10.593354 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f724b4-eb5d-4a2c-ac87-0b0e53290a80 None admin] PATH_INFO: `/v3/groups` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:10.593988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f724b4-eb5d-4a2c-ac87-0b0e53290a80 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:10.595454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f724b4-eb5d-4a2c-ac87-0b0e53290a80 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:10.615214 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b282143-7282-40fd-900e-91d1455ea0ed None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:10.650073 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2472b64-49fc-4f3b-8987-eadc60d30da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:10.650663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2472b64-49fc-4f3b-8987-eadc60d30da6 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:10.650936 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2472b64-49fc-4f3b-8987-eadc60d30da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:10.651233 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2472b64-49fc-4f3b-8987-eadc60d30da6 None admin] PATH_INFO: `/v3/users/d116c243fbcd4fbb980a3727f6ba46e7` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:10.652352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2472b64-49fc-4f3b-8987-eadc60d30da6 None admin] RBAC: Authorizing `identity:get_user(user_id=d116c243fbcd4fbb980a3727f6ba46e7)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:10.653779 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2472b64-49fc-4f3b-8987-eadc60d30da6 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:10.655380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 159/328] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:05:10 2022] GET /identity/v3/users/d116c243fbcd4fbb980a3727f6ba46e7 => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:10.661029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2472b64-49fc-4f3b-8987-eadc60d30da6 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:10.694168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 170/329] 10.222.0.78 () {60 vars in 1134 bytes} [Mon Sep 19 11:05:10 2022] POST /identity/v3/groups => generated 226 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:05:10.776543 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f724b4-eb5d-4a2c-ac87-0b0e53290a80 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:10.777136 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:10.780726 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2472b64-49fc-4f3b-8987-eadc60d30da6 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:10.826694 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed7a94-e9f4-490f-88d8-ae8deb6dc401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:10.827437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed7a94-e9f4-490f-88d8-ae8deb6dc401 None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:10.827721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed7a94-e9f4-490f-88d8-ae8deb6dc401 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:10.827997 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed7a94-e9f4-490f-88d8-ae8deb6dc401 None admin] PATH_INFO: `/v3/system/users/d116c243fbcd4fbb980a3727f6ba46e7/roles/05303606367c4086bb8e3c0c20d45b56` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:10.829505 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed7a94-e9f4-490f-88d8-ae8deb6dc401 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d116c243fbcd4fbb980a3727f6ba46e7, role_id=05303606367c4086bb8e3c0c20d45b56)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:10.831419 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed7a94-e9f4-490f-88d8-ae8deb6dc401 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:10.855703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:10.890818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f724b4-eb5d-4a2c-ac87-0b0e53290a80 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:10.904323 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 160/330] 10.222.0.78 () {58 vars in 1334 bytes} [Mon Sep 19 11:05:10 2022] PUT /identity/v3/system/users/d116c243fbcd4fbb980a3727f6ba46e7/roles/05303606367c4086bb8e3c0c20d45b56 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:05:10.932932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64488b12-17d3-4a9b-9cd8-82b467ea8937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:10.933604 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64488b12-17d3-4a9b-9cd8-82b467ea8937 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:10.933891 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64488b12-17d3-4a9b-9cd8-82b467ea8937 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:10.934167 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64488b12-17d3-4a9b-9cd8-82b467ea8937 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:10.947219 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64488b12-17d3-4a9b-9cd8-82b467ea8937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:10.952889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-64488b12-17d3-4a9b-9cd8-82b467ea8937 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:10.952889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:10.952889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:10.952889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:10.952889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:10.952889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:10.952889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:10.952889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:10.952889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:10.952889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:10.952889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:10.952889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:10.952889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:10.952889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:10.952889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:10.952889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:10.953923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:10.953923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:10.953923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:10.953923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:10.953923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:10.953923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:10.953923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:10.953923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:10.953923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:10.953923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:10.953923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:10.953923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:10.953923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:10.953923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:10.953923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:10.953923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:10.954749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:10.954749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:10.954749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:10.954749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:10.954749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:10.954749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:10.954749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:10.954749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:10.954749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:10.954749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:10.954749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:10.954749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:10.954749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 171/331] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:10.960116 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed7a94-e9f4-490f-88d8-ae8deb6dc401 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:11.086973 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed7a94-e9f4-490f-88d8-ae8deb6dc401 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:11.134777 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a53f0b-5ae8-4e78-8861-2a4ddbaa217a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:11.135464 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a53f0b-5ae8-4e78-8861-2a4ddbaa217a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:11.135742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a53f0b-5ae8-4e78-8861-2a4ddbaa217a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:11.136009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a53f0b-5ae8-4e78-8861-2a4ddbaa217a None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:11.136683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a53f0b-5ae8-4e78-8861-2a4ddbaa217a None admin] RBAC: Adding query filter params (name=member) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:11.136982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a53f0b-5ae8-4e78-8861-2a4ddbaa217a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:11.138356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a53f0b-5ae8-4e78-8861-2a4ddbaa217a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:11.149034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 161/332] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:05:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:11.155873 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64488b12-17d3-4a9b-9cd8-82b467ea8937 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:11.277032 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64488b12-17d3-4a9b-9cd8-82b467ea8937 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:11.304907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:11.315465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cb459-2bb4-4f72-aecf-d878265d1a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:11.316115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cb459-2bb4-4f72-aecf-d878265d1a9c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:11.323064 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cb459-2bb4-4f72-aecf-d878265d1a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:11.323064 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cb459-2bb4-4f72-aecf-d878265d1a9c None admin] PATH_INFO: `/v3/users/8ac302f871574494a2051e020256326a` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:11.323064 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cb459-2bb4-4f72-aecf-d878265d1a9c None admin] RBAC: Authorizing `identity:get_user(user_id=8ac302f871574494a2051e020256326a)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:11.323064 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cb459-2bb4-4f72-aecf-d878265d1a9c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:11.326252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 172/333] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:05:11 2022] GET /identity/v3/users/8ac302f871574494a2051e020256326a => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:11.327673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a53f0b-5ae8-4e78-8861-2a4ddbaa217a None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:11.358430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:11.454441 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a53f0b-5ae8-4e78-8861-2a4ddbaa217a None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:11.493780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ea519-e2bd-41d4-a277-d82f06bce32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:11.494774 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ea519-e2bd-41d4-a277-d82f06bce32c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:11.495355 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ea519-e2bd-41d4-a277-d82f06bce32c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:11.495836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ea519-e2bd-41d4-a277-d82f06bce32c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:11.498868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ea519-e2bd-41d4-a277-d82f06bce32c None admin] RBAC: Adding query filter params (role.id=486614b0ab4b454b92e9ef0bc7ec61c1, user.id=8ac302f871574494a2051e020256326a, scope.system=all) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:11.499472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ea519-e2bd-41d4-a277-d82f06bce32c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:11.501138 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ea519-e2bd-41d4-a277-d82f06bce32c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:11.514683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 162/334] 10.222.0.78 () {56 vars in 1308 bytes} [Mon Sep 19 11:05:11 2022] GET /identity/v3/role_assignments?user.id=8ac302f871574494a2051e020256326a&scope.system=all&role.id=486614b0ab4b454b92e9ef0bc7ec61c1 => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:13.281802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609503da-41f6-4f0e-a920-5a67a33cdf35 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:13.282303 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609503da-41f6-4f0e-a920-5a67a33cdf35 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:13.282404 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609503da-41f6-4f0e-a920-5a67a33cdf35 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:13.283988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 173/335] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:13.289635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92e93c-e689-4b9b-ab15-d35f799c4a9a None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:13.290416 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92e93c-e689-4b9b-ab15-d35f799c4a9a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:13.290862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92e93c-e689-4b9b-ab15-d35f799c4a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:13.316240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-4f92e93c-e689-4b9b-ab15-d35f799c4a9a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:13.317708 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4f92e93c-e689-4b9b-ab15-d35f799c4a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:13.375390 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a14eb0-dd8e-40cb-8d5b-d225529293eb None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:13.375905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a14eb0-dd8e-40cb-8d5b-d225529293eb None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:13.376359 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a14eb0-dd8e-40cb-8d5b-d225529293eb None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:13.378092 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 174/336] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:13.384709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052fd07-5236-410b-b062-d6bbd7cd771c None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:13.385270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052fd07-5236-410b-b062-d6bbd7cd771c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:13.385718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052fd07-5236-410b-b062-d6bbd7cd771c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:13.417971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 163/337] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:05:13.428331 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f052fd07-5236-410b-b062-d6bbd7cd771c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:13.430093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f052fd07-5236-410b-b062-d6bbd7cd771c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:13.525506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 175/338] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:05:13.817093 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7643152-bed3-4e9b-a89a-6aeac9a037db None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:13.817809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7643152-bed3-4e9b-a89a-6aeac9a037db None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:13.818291 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7643152-bed3-4e9b-a89a-6aeac9a037db None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:13.820097 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 164/339] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:13.837166 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebbe85-d93a-4862-8f20-9eab477d6cfa None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:13.837779 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebbe85-d93a-4862-8f20-9eab477d6cfa None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:13.838313 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebbe85-d93a-4862-8f20-9eab477d6cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:13.868151 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c0ebbe85-d93a-4862-8f20-9eab477d6cfa None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:13.870246 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c0ebbe85-d93a-4862-8f20-9eab477d6cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:13.956831 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 176/340] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:05:13.975955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7643152-bed3-4e9b-a89a-6aeac9a037db None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:13.998242 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:14.044120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebbe85-d93a-4862-8f20-9eab477d6cfa None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:14.111818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:14.155752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7643152-bed3-4e9b-a89a-6aeac9a037db None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:14.161615 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebbe85-d93a-4862-8f20-9eab477d6cfa None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:14.197713 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79146df1-540d-48d7-a085-ab715ce8b22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:14.198691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79146df1-540d-48d7-a085-ab715ce8b22d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:14.199149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79146df1-540d-48d7-a085-ab715ce8b22d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:14.199698 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79146df1-540d-48d7-a085-ab715ce8b22d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:14.211005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79146df1-540d-48d7-a085-ab715ce8b22d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:14.217412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-79146df1-540d-48d7-a085-ab715ce8b22d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:05:14.217412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:14.217412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:14.217412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:14.217412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:14.217412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:14.217412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:14.217412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:14.217412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:14.217412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:14.217412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:14.217412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:14.217412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:14.217412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:14.217412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:14.217412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:14.218287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:14.218287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:14.218287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:14.218287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:14.218287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:14.218287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:14.218287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:14.218287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:14.218287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:14.218287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:14.218287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:14.218287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:14.218287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:14.218287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:14.218287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:14.218287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:14.219077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:14.219077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:14.219077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:14.219077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:14.219077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:14.219077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:14.219077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:14.219077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:14.219077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:14.219077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:14.219077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:05:14.219077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:14.223579 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 177/341] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:05:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:14.224439 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79146df1-540d-48d7-a085-ab715ce8b22d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:14.224625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4a20d-3209-499d-bf67-7c7b97831cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:14.225377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4a20d-3209-499d-bf67-7c7b97831cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:14.225660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4a20d-3209-499d-bf67-7c7b97831cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:14.225930 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4a20d-3209-499d-bf67-7c7b97831cc0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:14.249542 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4a20d-3209-499d-bf67-7c7b97831cc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:14.254874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-00d4a20d-3209-499d-bf67-7c7b97831cc0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:05:14.254874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:14.254874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:14.254874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:14.254874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:14.254874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:14.254874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:14.254874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:14.254874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:14.254874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:14.254874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:14.254874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:14.254874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:14.254874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:14.254874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:14.254874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:14.260159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:14.260159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:14.260159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:14.260159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:14.260159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:14.260159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:14.260159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:14.260159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:14.260159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:14.260159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:14.260159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:14.260159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:14.260159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:14.260159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:14.260159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:14.260159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:14.261370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:14.261370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:14.261370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:14.261370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:14.261370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:14.261370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:14.261370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:14.261370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:14.261370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:14.261370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:14.261370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:05:14.261370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:14.261370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 165/342] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:14.264225 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4a20d-3209-499d-bf67-7c7b97831cc0 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:14.353007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79146df1-540d-48d7-a085-ab715ce8b22d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:14.376164 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:14.396750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:14.406088 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dba4a4-4fa1-46a7-a1e0-c9bf67f51b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:14.406720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dba4a4-4fa1-46a7-a1e0-c9bf67f51b05 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:14.407000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dba4a4-4fa1-46a7-a1e0-c9bf67f51b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:14.407294 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dba4a4-4fa1-46a7-a1e0-c9bf67f51b05 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:14.407969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dba4a4-4fa1-46a7-a1e0-c9bf67f51b05 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:14.408315 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dba4a4-4fa1-46a7-a1e0-c9bf67f51b05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:14.409660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dba4a4-4fa1-46a7-a1e0-c9bf67f51b05 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:14.418304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 178/343] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:14.425486 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dba4a4-4fa1-46a7-a1e0-c9bf67f51b05 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:14.436975 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4a20d-3209-499d-bf67-7c7b97831cc0 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:14.512483 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fa22c-7398-46aa-917c-3acac2d94dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:14.513114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fa22c-7398-46aa-917c-3acac2d94dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:14.513392 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fa22c-7398-46aa-917c-3acac2d94dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:14.513667 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fa22c-7398-46aa-917c-3acac2d94dc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:14.514355 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fa22c-7398-46aa-917c-3acac2d94dc3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:14.514654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fa22c-7398-46aa-917c-3acac2d94dc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:14.516085 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fa22c-7398-46aa-917c-3acac2d94dc3 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:14.535727 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dba4a4-4fa1-46a7-a1e0-c9bf67f51b05 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:14.537323 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 166/344] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:05:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:14.538729 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcca01-e892-4d4c-82e6-29b8d238a89d None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:14.539014 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcca01-e892-4d4c-82e6-29b8d238a89d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:14.542137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcca01-e892-4d4c-82e6-29b8d238a89d None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:14.543822 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 167/345] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:14.544616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcca01-e892-4d4c-82e6-29b8d238a89d None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:14.575851 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e423a-e427-4d0d-8ea7-f42032820d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:14.577043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e423a-e427-4d0d-8ea7-f42032820d87 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:14.577333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e423a-e427-4d0d-8ea7-f42032820d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:14.577672 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e423a-e427-4d0d-8ea7-f42032820d87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:14.646607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:14.677167 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e423a-e427-4d0d-8ea7-f42032820d87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:14.677843 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcca01-e892-4d4c-82e6-29b8d238a89d None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:14.678716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e423a-e427-4d0d-8ea7-f42032820d87 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:14.721528 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbdb2d-85d3-4b75-95c4-edd0c199fd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:14.722175 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbdb2d-85d3-4b75-95c4-edd0c199fd8c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:14.722443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbdb2d-85d3-4b75-95c4-edd0c199fd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:14.722716 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbdb2d-85d3-4b75-95c4-edd0c199fd8c None admin] PATH_INFO: `/v3/users/d116c243fbcd4fbb980a3727f6ba46e7` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:14.723872 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbdb2d-85d3-4b75-95c4-edd0c199fd8c None admin] RBAC: Authorizing `identity:get_user(user_id=d116c243fbcd4fbb980a3727f6ba46e7)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:14.725324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbdb2d-85d3-4b75-95c4-edd0c199fd8c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:14.727237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 168/346] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:05:14 2022] GET /identity/v3/users/d116c243fbcd4fbb980a3727f6ba46e7 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:14.734068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09508be-03d9-4952-9a41-adeb5a08860c None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:14.734383 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09508be-03d9-4952-9a41-adeb5a08860c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:14.734669 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09508be-03d9-4952-9a41-adeb5a08860c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:14.742632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:14.754789 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-b09508be-03d9-4952-9a41-adeb5a08860c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:14.756011 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b09508be-03d9-4952-9a41-adeb5a08860c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:14.806366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 179/347] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:14.808148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e423a-e427-4d0d-8ea7-f42032820d87 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:14.831620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 169/348] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:14.832498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09508be-03d9-4952-9a41-adeb5a08860c None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:14.919867 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:14.935877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09508be-03d9-4952-9a41-adeb5a08860c None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:14.965311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e423a-e427-4d0d-8ea7-f42032820d87 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:14.974186 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63560982-d921-41e5-a548-9f33d2ae1eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:14.974796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63560982-d921-41e5-a548-9f33d2ae1eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:14.975069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63560982-d921-41e5-a548-9f33d2ae1eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:14.975365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63560982-d921-41e5-a548-9f33d2ae1eb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:14.976173 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63560982-d921-41e5-a548-9f33d2ae1eb7 None admin] RBAC: Adding query filter params (role.id=05303606367c4086bb8e3c0c20d45b56, user.id=d116c243fbcd4fbb980a3727f6ba46e7, scope.system=all) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:14.976479 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63560982-d921-41e5-a548-9f33d2ae1eb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:14.977926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63560982-d921-41e5-a548-9f33d2ae1eb7 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:14.980999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:14.991472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 170/349] 10.222.0.78 () {56 vars in 1308 bytes} [Mon Sep 19 11:05:14 2022] GET /identity/v3/role_assignments?user.id=d116c243fbcd4fbb980a3727f6ba46e7&scope.system=all&role.id=05303606367c4086bb8e3c0c20d45b56 => generated 510 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:14.992247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63560982-d921-41e5-a548-9f33d2ae1eb7 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:15.043464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d870709-c7e0-4bec-a9ca-fa36eccbd40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:15.044136 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d870709-c7e0-4bec-a9ca-fa36eccbd40c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:15.044419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d870709-c7e0-4bec-a9ca-fa36eccbd40c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:15.044702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d870709-c7e0-4bec-a9ca-fa36eccbd40c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:15.067859 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d870709-c7e0-4bec-a9ca-fa36eccbd40c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:15.073747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-5d870709-c7e0-4bec-a9ca-fa36eccbd40c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:15.073747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:15.073747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:15.073747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:15.073747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:15.073747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:15.073747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:15.073747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:15.073747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:15.073747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:15.073747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:15.073747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:15.073747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:15.073747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:15.073747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:15.073747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:15.074695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:15.074695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:15.074695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:15.074695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:15.074695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:15.074695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:15.074695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:15.074695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:15.074695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:15.074695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:15.074695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:15.074695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:15.074695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:15.074695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:15.074695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:15.074695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:15.075559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:15.075559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:15.075559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:15.075559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:15.075559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:15.075559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:15.075559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:15.075559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:15.075559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:15.075559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:15.075559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:15.075559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:15.079386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 180/350] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:15.081611 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d870709-c7e0-4bec-a9ca-fa36eccbd40c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:15.136580 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63560982-d921-41e5-a548-9f33d2ae1eb7 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:15.179703 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a8f2d-1367-4a92-810c-d548acfab0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:15.180273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a8f2d-1367-4a92-810c-d548acfab0eb None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:15.180555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a8f2d-1367-4a92-810c-d548acfab0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:15.180828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a8f2d-1367-4a92-810c-d548acfab0eb None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:15.182600 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a8f2d-1367-4a92-810c-d548acfab0eb None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:15.184071 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a8f2d-1367-4a92-810c-d548acfab0eb None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:15.185741 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 171/351] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:14 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:15.200445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a8f2d-1367-4a92-810c-d548acfab0eb None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:15.287444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:15.336979 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d870709-c7e0-4bec-a9ca-fa36eccbd40c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:15.337334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a8f2d-1367-4a92-810c-d548acfab0eb None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:15.391752 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa2f99-1ae8-4fd2-a522-c8cd4348dc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:15.392379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa2f99-1ae8-4fd2-a522-c8cd4348dc41 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:15.392502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563df5af-bb02-433b-ab4f-a14726eebe97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:15.392675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa2f99-1ae8-4fd2-a522-c8cd4348dc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:15.392997 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa2f99-1ae8-4fd2-a522-c8cd4348dc41 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:15.393104 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563df5af-bb02-433b-ab4f-a14726eebe97 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:15.393467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563df5af-bb02-433b-ab4f-a14726eebe97 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:15.393631 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563df5af-bb02-433b-ab4f-a14726eebe97 None admin] PATH_INFO: `/v3/groups/366c65011d6f4485b7fac730fd7c4df6` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:15.393720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa2f99-1ae8-4fd2-a522-c8cd4348dc41 None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:15.394025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa2f99-1ae8-4fd2-a522-c8cd4348dc41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:15.395486 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa2f99-1ae8-4fd2-a522-c8cd4348dc41 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:15.401796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563df5af-bb02-433b-ab4f-a14726eebe97 None admin] RBAC: Authorizing `identity:get_group(group_id=366c65011d6f4485b7fac730fd7c4df6)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:15.405281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563df5af-bb02-433b-ab4f-a14726eebe97 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:15.405808 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:15.408798 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 172/352] 10.222.0.78 () {56 vars in 1180 bytes} [Mon Sep 19 11:05:15 2022] GET /identity/v3/groups/366c65011d6f4485b7fac730fd7c4df6 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:15.410062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d3751-22d8-4a53-ab0d-f5fc196a8f8d None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:15.410348 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d3751-22d8-4a53-ab0d-f5fc196a8f8d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:15.410623 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d3751-22d8-4a53-ab0d-f5fc196a8f8d None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:15.411893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 181/353] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:05:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:15.412149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 173/354] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:15.412731 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa2f99-1ae8-4fd2-a522-c8cd4348dc41 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:15.417753 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e4213-365d-46bf-bb88-feb363200b83 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:15.418043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e4213-365d-46bf-bb88-feb363200b83 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:15.418420 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e4213-365d-46bf-bb88-feb363200b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:15.444324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-ea3e4213-365d-46bf-bb88-feb363200b83 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:15.445897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ea3e4213-365d-46bf-bb88-feb363200b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:15.539235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 174/355] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:05:15.540149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e4213-365d-46bf-bb88-feb363200b83 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:15.545765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa2f99-1ae8-4fd2-a522-c8cd4348dc41 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:15.605710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:15.627989 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea8cb9-d528-401a-8481-81a69b881535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:15.629325 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea8cb9-d528-401a-8481-81a69b881535 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:15.630279 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea8cb9-d528-401a-8481-81a69b881535 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:15.630802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea8cb9-d528-401a-8481-81a69b881535 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:15.646950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea8cb9-d528-401a-8481-81a69b881535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:15.655822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-70ea8cb9-d528-401a-8481-81a69b881535 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:15.655822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:15.655822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:15.655822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:15.655822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:15.655822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:15.655822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:15.655822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:15.655822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:15.655822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:15.655822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:15.655822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:15.655822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:15.655822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:15.655822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:15.655822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:15.656744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:15.656744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:15.656744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:15.656744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:15.656744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:15.656744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:15.656744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:15.656744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:15.656744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:15.656744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:15.656744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:15.656744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:15.656744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:15.656744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:15.656744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:15.656744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:15.657520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:15.657520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:15.657520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:15.657520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:15.657520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:15.657520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:15.657520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:15.657520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:15.657520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:15.657520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:15.657520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:15.657520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:15.660067 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 182/356] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:15.661264 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea8cb9-d528-401a-8481-81a69b881535 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:15.666302 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e4213-365d-46bf-bb88-feb363200b83 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:15.705605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71efef5-900a-44b6-96c2-02795bd33308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:15.706239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71efef5-900a-44b6-96c2-02795bd33308 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:15.706622 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71efef5-900a-44b6-96c2-02795bd33308 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:15.706998 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71efef5-900a-44b6-96c2-02795bd33308 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:15.745149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:15.801878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71efef5-900a-44b6-96c2-02795bd33308 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:15.804828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71efef5-900a-44b6-96c2-02795bd33308 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:15.810325 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea8cb9-d528-401a-8481-81a69b881535 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:15.848790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89af8e3-65ba-4ef7-b963-76b1b39ce240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:15.849416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89af8e3-65ba-4ef7-b963-76b1b39ce240 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:15.849701 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89af8e3-65ba-4ef7-b963-76b1b39ce240 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:15.849980 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89af8e3-65ba-4ef7-b963-76b1b39ce240 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:15.850717 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89af8e3-65ba-4ef7-b963-76b1b39ce240 None admin] RBAC: Adding query filter params (group.id=366c65011d6f4485b7fac730fd7c4df6, role.id=b7eccd60e28d4b39b54879dad8230e0d, scope.project.id=ff6a8ae52b074b64be28a144288b4780) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:15.851030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89af8e3-65ba-4ef7-b963-76b1b39ce240 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:15.852526 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89af8e3-65ba-4ef7-b963-76b1b39ce240 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:15.859456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 183/357] 10.222.0.78 () {56 vars in 1376 bytes} [Mon Sep 19 11:05:15 2022] GET /identity/v3/role_assignments?group.id=366c65011d6f4485b7fac730fd7c4df6&scope.project.id=ff6a8ae52b074b64be28a144288b4780&role.id=b7eccd60e28d4b39b54879dad8230e0d => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:15.860594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89af8e3-65ba-4ef7-b963-76b1b39ce240 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:15.891567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:15.952241 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 175/358] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:15.952949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71efef5-900a-44b6-96c2-02795bd33308 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:16.005665 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89af8e3-65ba-4ef7-b963-76b1b39ce240 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:16.042833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:16.066095 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ed486-e655-420a-913e-641786fe62ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:16.066730 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ed486-e655-420a-913e-641786fe62ae None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:16.067078 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ed486-e655-420a-913e-641786fe62ae None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:16.068652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ed486-e655-420a-913e-641786fe62ae None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:16.069351 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ed486-e655-420a-913e-641786fe62ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:16.069724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ed486-e655-420a-913e-641786fe62ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:16.072367 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ed486-e655-420a-913e-641786fe62ae None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:16.085793 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 184/359] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:05:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:16.090845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ed486-e655-420a-913e-641786fe62ae None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:16.098769 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71efef5-900a-44b6-96c2-02795bd33308 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:16.148088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fec57-2eea-4cfd-ab4e-d0a5836c01dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:16.148988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fec57-2eea-4cfd-ab4e-d0a5836c01dd None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:16.149265 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fec57-2eea-4cfd-ab4e-d0a5836c01dd None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:16.149538 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fec57-2eea-4cfd-ab4e-d0a5836c01dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:16.162818 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839fec57-2eea-4cfd-ab4e-d0a5836c01dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:16.169635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-839fec57-2eea-4cfd-ab4e-d0a5836c01dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:05:16.169635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:16.169635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:16.169635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:16.169635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:16.169635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:16.169635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:16.169635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:16.169635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:16.169635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:16.169635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:16.169635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:16.169635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:16.169635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:16.169635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:16.169635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:16.170616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:16.170616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:16.170616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:16.170616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:16.170616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:16.170616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:16.170616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:16.170616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:16.170616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:16.170616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:16.170616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:16.170616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:16.170616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:16.170616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:16.170616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:16.170616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:16.171458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:16.171458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:16.171458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:16.171458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:16.171458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:16.171458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:16.171458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:16.171458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:16.171458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:16.171458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:16.171458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:05:16.171458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:16.171458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 176/360] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:05:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:16.171458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fec57-2eea-4cfd-ab4e-d0a5836c01dd None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:16.193372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:16.242413 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ed486-e655-420a-913e-641786fe62ae None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:16.282266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd00f2-e733-4b81-8161-d238064ee97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:16.282837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd00f2-e733-4b81-8161-d238064ee97b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:16.283117 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd00f2-e733-4b81-8161-d238064ee97b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:16.283742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd00f2-e733-4b81-8161-d238064ee97b None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:16.284656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd00f2-e733-4b81-8161-d238064ee97b None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:16.286148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd00f2-e733-4b81-8161-d238064ee97b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:16.291673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 185/361] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:16 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:16.292694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd00f2-e733-4b81-8161-d238064ee97b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:16.311961 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fec57-2eea-4cfd-ab4e-d0a5836c01dd None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:16.350936 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434a7ef-d543-4e96-a1d1-482e41f71df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:16.351931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434a7ef-d543-4e96-a1d1-482e41f71df0 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:16.352416 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434a7ef-d543-4e96-a1d1-482e41f71df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:16.352892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434a7ef-d543-4e96-a1d1-482e41f71df0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:16.354403 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:16.417500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd00f2-e733-4b81-8161-d238064ee97b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:16.453786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434a7ef-d543-4e96-a1d1-482e41f71df0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:16.455957 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434a7ef-d543-4e96-a1d1-482e41f71df0 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:16.467402 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd01021-ffef-41b7-9988-7b80eaa85202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:16.468131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd01021-ffef-41b7-9988-7b80eaa85202 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:16.468488 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd01021-ffef-41b7-9988-7b80eaa85202 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:16.468949 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd01021-ffef-41b7-9988-7b80eaa85202 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:16.469641 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd01021-ffef-41b7-9988-7b80eaa85202 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:16.469971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd01021-ffef-41b7-9988-7b80eaa85202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:16.471354 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd01021-ffef-41b7-9988-7b80eaa85202 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:16.473396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:16.482149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 186/362] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:16.483418 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd01021-ffef-41b7-9988-7b80eaa85202 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:16.602860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 177/363] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:16.606285 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68164a-aa28-478b-9a6a-612fa5698882 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:16.606829 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68164a-aa28-478b-9a6a-612fa5698882 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:16.607353 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68164a-aa28-478b-9a6a-612fa5698882 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:16.608932 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 178/364] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:16.609891 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68164a-aa28-478b-9a6a-612fa5698882 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:16.618945 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd01021-ffef-41b7-9988-7b80eaa85202 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:16.639936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:16.675810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649d897-6aa4-440e-9d85-c1b60fd66165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:16.676476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649d897-6aa4-440e-9d85-c1b60fd66165 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:16.676763 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649d897-6aa4-440e-9d85-c1b60fd66165 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:16.677043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649d897-6aa4-440e-9d85-c1b60fd66165 None admin] PATH_INFO: `/v3/groups/366c65011d6f4485b7fac730fd7c4df6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:16.678335 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649d897-6aa4-440e-9d85-c1b60fd66165 None admin] RBAC: Authorizing `identity:get_group(group_id=366c65011d6f4485b7fac730fd7c4df6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:16.680221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649d897-6aa4-440e-9d85-c1b60fd66165 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:16.683767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 187/365] 10.222.0.78 () {56 vars in 1180 bytes} [Mon Sep 19 11:05:16 2022] GET /identity/v3/groups/366c65011d6f4485b7fac730fd7c4df6 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:16.684024 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649d897-6aa4-440e-9d85-c1b60fd66165 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:16.765261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68164a-aa28-478b-9a6a-612fa5698882 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:16.785970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:16.814071 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649d897-6aa4-440e-9d85-c1b60fd66165 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:16.817276 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd1b53-812b-4387-8f6d-ded28ae36ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:16.817903 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd1b53-812b-4387-8f6d-ded28ae36ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:16.818179 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd1b53-812b-4387-8f6d-ded28ae36ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:16.818458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd1b53-812b-4387-8f6d-ded28ae36ae4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:16.866328 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa60d59-b7fb-4a43-8065-c24468e93628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:16.867337 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa60d59-b7fb-4a43-8065-c24468e93628 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:16.867805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa60d59-b7fb-4a43-8065-c24468e93628 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:16.868262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa60d59-b7fb-4a43-8065-c24468e93628 None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:16.870966 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa60d59-b7fb-4a43-8065-c24468e93628 None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:16.873363 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa60d59-b7fb-4a43-8065-c24468e93628 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:16.875205 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 188/366] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:16 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:16.876899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9dd6a-9b38-4379-8850-bbb363d984f0 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:16.877189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9dd6a-9b38-4379-8850-bbb363d984f0 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:16.877462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9dd6a-9b38-4379-8850-bbb363d984f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:16.904043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d2a9dd6a-9b38-4379-8850-bbb363d984f0 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:16.905251 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d2a9dd6a-9b38-4379-8850-bbb363d984f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:16.906814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd1b53-812b-4387-8f6d-ded28ae36ae4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:16.908324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd1b53-812b-4387-8f6d-ded28ae36ae4 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:16.960289 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:16.994191 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 189/367] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:16.995081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9dd6a-9b38-4379-8850-bbb363d984f0 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:17.027217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 179/368] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:17.028143 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd1b53-812b-4387-8f6d-ded28ae36ae4 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:17.047953 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:17.139570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9dd6a-9b38-4379-8850-bbb363d984f0 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:17.158459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd1b53-812b-4387-8f6d-ded28ae36ae4 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:17.192523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3982b0b-7e0f-4bfd-9e1b-2324fd960421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:17.193155 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3982b0b-7e0f-4bfd-9e1b-2324fd960421 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:17.193545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3982b0b-7e0f-4bfd-9e1b-2324fd960421 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:17.193714 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3982b0b-7e0f-4bfd-9e1b-2324fd960421 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:17.194446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3982b0b-7e0f-4bfd-9e1b-2324fd960421 None admin] RBAC: Adding query filter params (group.id=366c65011d6f4485b7fac730fd7c4df6, role.id=486614b0ab4b454b92e9ef0bc7ec61c1, scope.project.id=ff6a8ae52b074b64be28a144288b4780) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:17.194754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3982b0b-7e0f-4bfd-9e1b-2324fd960421 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:17.196235 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3982b0b-7e0f-4bfd-9e1b-2324fd960421 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:17.204081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 190/369] 10.222.0.78 () {56 vars in 1376 bytes} [Mon Sep 19 11:05:16 2022] GET /identity/v3/role_assignments?group.id=366c65011d6f4485b7fac730fd7c4df6&scope.project.id=ff6a8ae52b074b64be28a144288b4780&role.id=486614b0ab4b454b92e9ef0bc7ec61c1 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:17.204256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3982b0b-7e0f-4bfd-9e1b-2324fd960421 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:17.215972 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d2043-878f-42aa-982d-7fd3b34438fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:17.216966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d2043-878f-42aa-982d-7fd3b34438fe None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:17.217422 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d2043-878f-42aa-982d-7fd3b34438fe None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:17.217869 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d2043-878f-42aa-982d-7fd3b34438fe None admin] PATH_INFO: `/v3/groups/366c65011d6f4485b7fac730fd7c4df6` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:17.219438 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d2043-878f-42aa-982d-7fd3b34438fe None admin] RBAC: Authorizing `identity:get_group(group_id=366c65011d6f4485b7fac730fd7c4df6)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:17.221207 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d2043-878f-42aa-982d-7fd3b34438fe None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:17.224388 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 180/370] 10.222.0.78 () {56 vars in 1180 bytes} [Mon Sep 19 11:05:17 2022] GET /identity/v3/groups/366c65011d6f4485b7fac730fd7c4df6 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:17.229907 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d2043-878f-42aa-982d-7fd3b34438fe None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:17.273011 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:17.369043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3982b0b-7e0f-4bfd-9e1b-2324fd960421 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:17.390044 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:17.419243 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8b2f4-7449-4ffa-a029-c55e0f5ad324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:17.420009 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8b2f4-7449-4ffa-a029-c55e0f5ad324 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:17.420364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8b2f4-7449-4ffa-a029-c55e0f5ad324 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:17.420894 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8b2f4-7449-4ffa-a029-c55e0f5ad324 None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:17.422754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8b2f4-7449-4ffa-a029-c55e0f5ad324 None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:17.423484 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d2043-878f-42aa-982d-7fd3b34438fe None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:17.424545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8b2f4-7449-4ffa-a029-c55e0f5ad324 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:17.426832 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 191/371] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:17 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:17.428893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8b2f4-7449-4ffa-a029-c55e0f5ad324 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:17.478527 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03285b7-15d4-4730-8614-c8e98ff3edab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:17.479833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03285b7-15d4-4730-8614-c8e98ff3edab None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:17.480303 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03285b7-15d4-4730-8614-c8e98ff3edab None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:17.480802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03285b7-15d4-4730-8614-c8e98ff3edab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:17.481696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03285b7-15d4-4730-8614-c8e98ff3edab None admin] RBAC: Adding query filter params (group.id=366c65011d6f4485b7fac730fd7c4df6, role.id=486614b0ab4b454b92e9ef0bc7ec61c1, scope.project.id=1453ab88d2354019a189c0e0883fd8b1) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:17.482174 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03285b7-15d4-4730-8614-c8e98ff3edab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:17.484430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03285b7-15d4-4730-8614-c8e98ff3edab None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:17.493140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 181/372] 10.222.0.78 () {56 vars in 1376 bytes} [Mon Sep 19 11:05:17 2022] GET /identity/v3/role_assignments?group.id=366c65011d6f4485b7fac730fd7c4df6&scope.project.id=1453ab88d2354019a189c0e0883fd8b1&role.id=486614b0ab4b454b92e9ef0bc7ec61c1 => generated 260 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:17.494955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03285b7-15d4-4730-8614-c8e98ff3edab None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:17.611680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8b2f4-7449-4ffa-a029-c55e0f5ad324 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:17.632861 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:17.661777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f226c-93de-4022-98ec-682f232a0307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:17.662574 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f226c-93de-4022-98ec-682f232a0307 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:17.663107 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f226c-93de-4022-98ec-682f232a0307 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:17.663419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f226c-93de-4022-98ec-682f232a0307 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:17.677611 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5f226c-93de-4022-98ec-682f232a0307 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:17.686519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-7f5f226c-93de-4022-98ec-682f232a0307 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:05:17.686519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:17.686519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:17.686519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:17.686519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:17.686519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:17.686519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:17.686519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:17.686519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:17.686519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:17.686519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:17.686519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:17.686519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:17.686519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:17.686519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:17.686519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:17.687565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:17.687565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:17.687565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:17.687565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:17.687565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:17.687565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:17.687565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:17.687565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:17.687565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:17.687565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:17.687565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:17.687565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:17.687565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:17.687565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:17.687565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:17.687565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:17.688854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:17.688854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:17.688854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:17.688854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:17.688854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:17.688854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:17.688854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:17.688854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:17.688854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:17.688854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:17.688854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:05:17.688854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:17.688854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 192/373] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:05:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:17.696655 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f226c-93de-4022-98ec-682f232a0307 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:17.745963 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03285b7-15d4-4730-8614-c8e98ff3edab None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:17.786841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a63cf7-25a1-462d-a0ee-0e300e8850c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:17.787565 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a63cf7-25a1-462d-a0ee-0e300e8850c3 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:17.787852 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a63cf7-25a1-462d-a0ee-0e300e8850c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:17.788129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a63cf7-25a1-462d-a0ee-0e300e8850c3 None admin] PATH_INFO: `/v3/groups/366c65011d6f4485b7fac730fd7c4df6` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:17.789500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a63cf7-25a1-462d-a0ee-0e300e8850c3 None admin] RBAC: Authorizing `identity:get_group(group_id=366c65011d6f4485b7fac730fd7c4df6)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:17.791099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a63cf7-25a1-462d-a0ee-0e300e8850c3 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:17.793923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 182/374] 10.222.0.78 () {56 vars in 1180 bytes} [Mon Sep 19 11:05:17 2022] GET /identity/v3/groups/366c65011d6f4485b7fac730fd7c4df6 => generated 226 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:17.799893 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a63cf7-25a1-462d-a0ee-0e300e8850c3 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:17.820072 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:17.853093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f226c-93de-4022-98ec-682f232a0307 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:17.897907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01146c81-bd9b-4970-a11c-391e9a049f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:17.898864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01146c81-bd9b-4970-a11c-391e9a049f80 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:17.899394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01146c81-bd9b-4970-a11c-391e9a049f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:17.899873 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01146c81-bd9b-4970-a11c-391e9a049f80 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:17.901005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01146c81-bd9b-4970-a11c-391e9a049f80 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:17.901525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01146c81-bd9b-4970-a11c-391e9a049f80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:17.903594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01146c81-bd9b-4970-a11c-391e9a049f80 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:17.914478 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 193/375] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:17.921560 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a63cf7-25a1-462d-a0ee-0e300e8850c3 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:17.924087 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01146c81-bd9b-4970-a11c-391e9a049f80 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:17.995566 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:18.009389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ab1d5-26ae-405c-9372-027f29ddd91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:18.011200 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ab1d5-26ae-405c-9372-027f29ddd91a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:18.011200 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ab1d5-26ae-405c-9372-027f29ddd91a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:18.011200 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ab1d5-26ae-405c-9372-027f29ddd91a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:18.011859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ab1d5-26ae-405c-9372-027f29ddd91a None admin] RBAC: Adding query filter params (group.id=366c65011d6f4485b7fac730fd7c4df6, role.id=b7eccd60e28d4b39b54879dad8230e0d, scope.project.id=1453ab88d2354019a189c0e0883fd8b1) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:18.012171 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ab1d5-26ae-405c-9372-027f29ddd91a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:18.013630 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ab1d5-26ae-405c-9372-027f29ddd91a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:18.037500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 183/376] 10.222.0.78 () {56 vars in 1376 bytes} [Mon Sep 19 11:05:17 2022] GET /identity/v3/role_assignments?group.id=366c65011d6f4485b7fac730fd7c4df6&scope.project.id=1453ab88d2354019a189c0e0883fd8b1&role.id=b7eccd60e28d4b39b54879dad8230e0d => generated 260 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:18.082712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01146c81-bd9b-4970-a11c-391e9a049f80 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:18.143146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9946a4-486c-446a-9c21-ca1c7c2adc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:18.143818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9946a4-486c-446a-9c21-ca1c7c2adc39 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:18.144132 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9946a4-486c-446a-9c21-ca1c7c2adc39 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:18.144442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9946a4-486c-446a-9c21-ca1c7c2adc39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:18.231407 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9946a4-486c-446a-9c21-ca1c7c2adc39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:18.232992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9946a4-486c-446a-9c21-ca1c7c2adc39 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:18.272039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:18.338178 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1c606-0cab-4232-a5b4-c7c4ac4c8eb7 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:18.338178 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1c606-0cab-4232-a5b4-c7c4ac4c8eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:18.338459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1c606-0cab-4232-a5b4-c7c4ac4c8eb7 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:18.339801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 184/377] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:18.346682 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d51a2-3f5d-4322-8588-feac398d5149 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:18.346999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d51a2-3f5d-4322-8588-feac398d5149 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:18.347267 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d51a2-3f5d-4322-8588-feac398d5149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:18.352706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 194/378] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:18.360266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9946a4-486c-446a-9c21-ca1c7c2adc39 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:18.375198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-a77d51a2-3f5d-4322-8588-feac398d5149 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:18.375198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a77d51a2-3f5d-4322-8588-feac398d5149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:18.485728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 185/379] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:05:18.526858 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9946a4-486c-446a-9c21-ca1c7c2adc39 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:18.565517 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad2ab3-2f04-474f-a840-91912386fcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:18.566490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad2ab3-2f04-474f-a840-91912386fcb8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:18.566929 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad2ab3-2f04-474f-a840-91912386fcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:18.567386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad2ab3-2f04-474f-a840-91912386fcb8 None admin] PATH_INFO: `/v3/projects/a875b13c6f2b4a43a97ce59e85aa99fd` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:18.568691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad2ab3-2f04-474f-a840-91912386fcb8 None admin] RBAC: Authorizing `identity:get_project(project_id=a875b13c6f2b4a43a97ce59e85aa99fd)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:18.570302 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad2ab3-2f04-474f-a840-91912386fcb8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:18.572491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 195/380] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:18 2022] GET /identity/v3/projects/a875b13c6f2b4a43a97ce59e85aa99fd => generated 344 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:18.578440 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d51a2-3f5d-4322-8588-feac398d5149 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:18.593912 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:18.707773 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d51a2-3f5d-4322-8588-feac398d5149 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:18.745504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ea338-4bc7-4d41-b158-695e694852b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:18.746062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ea338-4bc7-4d41-b158-695e694852b7 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:18.746323 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ea338-4bc7-4d41-b158-695e694852b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:18.746594 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ea338-4bc7-4d41-b158-695e694852b7 None admin] PATH_INFO: `/v3/groups/0335f2cbdb24418ebf1b1a26dae9f7e5` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:18.753591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ea338-4bc7-4d41-b158-695e694852b7 None admin] RBAC: Authorizing `identity:get_group(group_id=0335f2cbdb24418ebf1b1a26dae9f7e5)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:18.755421 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ea338-4bc7-4d41-b158-695e694852b7 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:18.757607 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 186/381] 10.222.0.78 () {56 vars in 1180 bytes} [Mon Sep 19 11:05:18 2022] GET /identity/v3/groups/0335f2cbdb24418ebf1b1a26dae9f7e5 => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:18.763715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad2ab3-2f04-474f-a840-91912386fcb8 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:18.768704 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:18.822913 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ea338-4bc7-4d41-b158-695e694852b7 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:18.889286 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad2ab3-2f04-474f-a840-91912386fcb8 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:18.912740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:18.929657 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174f057-3351-4eb7-aef4-9604dba9e3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:18.930603 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174f057-3351-4eb7-aef4-9604dba9e3eb None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:18.931065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174f057-3351-4eb7-aef4-9604dba9e3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:18.931621 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174f057-3351-4eb7-aef4-9604dba9e3eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:18.932544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174f057-3351-4eb7-aef4-9604dba9e3eb None admin] RBAC: Adding query filter params (group.id=0335f2cbdb24418ebf1b1a26dae9f7e5, role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=a875b13c6f2b4a43a97ce59e85aa99fd) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:18.933051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174f057-3351-4eb7-aef4-9604dba9e3eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:18.934661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174f057-3351-4eb7-aef4-9604dba9e3eb None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:18.941118 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 196/382] 10.222.0.78 () {56 vars in 1376 bytes} [Mon Sep 19 11:05:18 2022] GET /identity/v3/role_assignments?group.id=0335f2cbdb24418ebf1b1a26dae9f7e5&scope.project.id=a875b13c6f2b4a43a97ce59e85aa99fd&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:18.949172 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ea338-4bc7-4d41-b158-695e694852b7 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:18.996118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4249d9-fcea-4fce-a92d-7c4ed3faac15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:18.997095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4249d9-fcea-4fce-a92d-7c4ed3faac15 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:18.997556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4249d9-fcea-4fce-a92d-7c4ed3faac15 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:18.998005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4249d9-fcea-4fce-a92d-7c4ed3faac15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:19.014485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4249d9-fcea-4fce-a92d-7c4ed3faac15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:19.020028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-5c4249d9-fcea-4fce-a92d-7c4ed3faac15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:05:19.020028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:19.020028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:19.020028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:19.020028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:19.020028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:19.020028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:19.020028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:19.020028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:19.020028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:19.020028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:19.020028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:19.020028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:19.020028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:19.020028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:19.020028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:19.021000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:19.021000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:19.021000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:19.021000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:19.021000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:19.021000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:19.021000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:19.021000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:19.021000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:19.021000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:19.021000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:19.021000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:19.021000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:19.021000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:19.021000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:19.021000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:19.021833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:19.021833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:19.021833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:19.021833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:19.021833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:19.021833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:19.021833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:19.021833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:19.021833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:19.021833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:19.021833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:05:19.021833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:19.024350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 187/383] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:05:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:19.031420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174f057-3351-4eb7-aef4-9604dba9e3eb None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:19.116369 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:19.155841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174f057-3351-4eb7-aef4-9604dba9e3eb None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:19.200702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee7137-2dd7-4e70-bb0d-1e78003783b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:19.201638 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee7137-2dd7-4e70-bb0d-1e78003783b0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:19.202080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee7137-2dd7-4e70-bb0d-1e78003783b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:19.205217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee7137-2dd7-4e70-bb0d-1e78003783b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:19.206207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee7137-2dd7-4e70-bb0d-1e78003783b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:19.206676 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee7137-2dd7-4e70-bb0d-1e78003783b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:19.208283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee7137-2dd7-4e70-bb0d-1e78003783b0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:19.218267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 197/384] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:19.224434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4249d9-fcea-4fce-a92d-7c4ed3faac15 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:19.347736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4249d9-fcea-4fce-a92d-7c4ed3faac15 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:19.384995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:19.390723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f75f5-da12-42bf-ac68-d0c1052a5f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:19.391443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f75f5-da12-42bf-ac68-d0c1052a5f36 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:19.397049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f75f5-da12-42bf-ac68-d0c1052a5f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:19.397049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f75f5-da12-42bf-ac68-d0c1052a5f36 None admin] PATH_INFO: `/v3/groups/366c65011d6f4485b7fac730fd7c4df6` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:19.397049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f75f5-da12-42bf-ac68-d0c1052a5f36 None admin] RBAC: Authorizing `identity:get_group(group_id=366c65011d6f4485b7fac730fd7c4df6)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:19.397049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f75f5-da12-42bf-ac68-d0c1052a5f36 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:19.397049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 188/385] 10.222.0.78 () {56 vars in 1180 bytes} [Mon Sep 19 11:05:19 2022] GET /identity/v3/groups/366c65011d6f4485b7fac730fd7c4df6 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:19.402076 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee7137-2dd7-4e70-bb0d-1e78003783b0 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:19.513949 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee7137-2dd7-4e70-bb0d-1e78003783b0 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:19.545495 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:19.551518 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2702e-68f0-4aa9-a2d5-244a77519058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:19.557172 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2702e-68f0-4aa9-a2d5-244a77519058 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:19.557172 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2702e-68f0-4aa9-a2d5-244a77519058 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:19.557172 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2702e-68f0-4aa9-a2d5-244a77519058 None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:19.557172 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2702e-68f0-4aa9-a2d5-244a77519058 None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:19.557172 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2702e-68f0-4aa9-a2d5-244a77519058 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:19.558865 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 198/386] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:19 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:19.565741 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f75f5-da12-42bf-ac68-d0c1052a5f36 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:19.685718 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f75f5-da12-42bf-ac68-d0c1052a5f36 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:19.725061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404987ad-dd62-4dce-8e78-4baeb9e526b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:19.725740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404987ad-dd62-4dce-8e78-4baeb9e526b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:19.726017 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404987ad-dd62-4dce-8e78-4baeb9e526b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:19.726289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404987ad-dd62-4dce-8e78-4baeb9e526b8 None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780/groups/366c65011d6f4485b7fac730fd7c4df6/roles/b7eccd60e28d4b39b54879dad8230e0d` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:19.732428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404987ad-dd62-4dce-8e78-4baeb9e526b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff6a8ae52b074b64be28a144288b4780, group_id=366c65011d6f4485b7fac730fd7c4df6, role_id=b7eccd60e28d4b39b54879dad8230e0d)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:19.733879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404987ad-dd62-4dce-8e78-4baeb9e526b8 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:19.812244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 189/387] 10.222.0.78 () {58 vars in 1442 bytes} [Mon Sep 19 11:05:19 2022] PUT /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780/groups/366c65011d6f4485b7fac730fd7c4df6/roles/b7eccd60e28d4b39b54879dad8230e0d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:05:20.553723 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc0b8f-db42-47f9-9769-75819705b6d0 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:20.554270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc0b8f-db42-47f9-9769-75819705b6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:20.554270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc0b8f-db42-47f9-9769-75819705b6d0 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:20.555898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 199/388] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:20.560986 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c072ea2-9adb-4638-a9d1-876dc9d0b76d None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:20.561274 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c072ea2-9adb-4638-a9d1-876dc9d0b76d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:20.561546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c072ea2-9adb-4638-a9d1-876dc9d0b76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:20.592277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-7c072ea2-9adb-4638-a9d1-876dc9d0b76d None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:20.593487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7c072ea2-9adb-4638-a9d1-876dc9d0b76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:20.659757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:20.673105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 190/389] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:20.687839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7674a-a9f6-4f49-b360-8bbe4c53ae91 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:20.688454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7674a-a9f6-4f49-b360-8bbe4c53ae91 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:20.688937 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7674a-a9f6-4f49-b360-8bbe4c53ae91 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:20.690782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 200/390] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:20.696403 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e249a-4331-4b23-b524-1d5552186ced None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:20.697009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e249a-4331-4b23-b524-1d5552186ced None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:20.701155 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e249a-4331-4b23-b524-1d5552186ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:20.720244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-4f6e249a-4331-4b23-b524-1d5552186ced None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:20.721730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4f6e249a-4331-4b23-b524-1d5552186ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:20.807976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 191/391] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:20.888236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea42c9-20d4-4350-967c-01d96d58aac0 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:20.888582 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea42c9-20d4-4350-967c-01d96d58aac0 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:20.888797 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea42c9-20d4-4350-967c-01d96d58aac0 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:20.890274 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 201/392] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:20.895788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee4c1a-5292-4d91-a0b0-4461bdb4f6b5 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:20.896372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee4c1a-5292-4d91-a0b0-4461bdb4f6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:20.896874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee4c1a-5292-4d91-a0b0-4461bdb4f6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:20.915999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-97ee4c1a-5292-4d91-a0b0-4461bdb4f6b5 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:20.917572 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-97ee4c1a-5292-4d91-a0b0-4461bdb4f6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:21.002846 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 192/393] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:21.007340 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea42c9-20d4-4350-967c-01d96d58aac0 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:21.142084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea42c9-20d4-4350-967c-01d96d58aac0 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:21.169354 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee4c1a-5292-4d91-a0b0-4461bdb4f6b5 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:21.188435 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3901abf-2b12-4fbc-865b-ac9894ba4492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:21.189082 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3901abf-2b12-4fbc-865b-ac9894ba4492 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:21.189362 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3901abf-2b12-4fbc-865b-ac9894ba4492 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:21.189633 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3901abf-2b12-4fbc-865b-ac9894ba4492 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:21.201293 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3901abf-2b12-4fbc-865b-ac9894ba4492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:21.211204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-e3901abf-2b12-4fbc-865b-ac9894ba4492 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:21.211204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:21.211204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:21.211204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:21.211204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:21.211204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:21.211204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:21.211204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:21.211204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:21.211204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:21.211204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:21.211204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:21.211204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:21.211204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:21.211204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:21.211204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:21.212158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:21.212158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:21.212158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:21.212158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:21.212158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:21.212158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:21.212158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:21.212158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:21.212158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:21.212158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:21.212158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:21.212158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:21.212158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:21.212158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:21.212158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:21.212158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:21.212978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:21.212978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:21.212978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:21.212978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:21.212978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:21.212978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:21.212978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:21.212978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:21.212978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:21.212978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:21.212978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:21.212978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:21.212978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 202/394] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:21.214427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3901abf-2b12-4fbc-865b-ac9894ba4492 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:21.279846 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:21.343660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:21.372038 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee4c1a-5292-4d91-a0b0-4461bdb4f6b5 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:21.390714 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3901abf-2b12-4fbc-865b-ac9894ba4492 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:21.419820 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2f8e5-0d3b-422c-9003-be0e1ffe5f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:21.420933 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2f8e5-0d3b-422c-9003-be0e1ffe5f90 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:21.421439 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2f8e5-0d3b-422c-9003-be0e1ffe5f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:21.421926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2f8e5-0d3b-422c-9003-be0e1ffe5f90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:21.436889 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec2f8e5-0d3b-422c-9003-be0e1ffe5f90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:21.438938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f2f13-0083-4031-bba1-dc58a2918784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:21.439577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f2f13-0083-4031-bba1-dc58a2918784 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:21.439856 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f2f13-0083-4031-bba1-dc58a2918784 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:21.440127 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f2f13-0083-4031-bba1-dc58a2918784 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:21.440795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f2f13-0083-4031-bba1-dc58a2918784 None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:21.441096 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f2f13-0083-4031-bba1-dc58a2918784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:21.442525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f2f13-0083-4031-bba1-dc58a2918784 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:21.444118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-6ec2f8e5-0d3b-422c-9003-be0e1ffe5f90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:21.444118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:21.444118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:21.444118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:21.444118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:21.444118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:21.444118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:21.444118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:21.444118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:21.444118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:21.444118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:21.444118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:21.444118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:21.444118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:21.444118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:21.444118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:21.445080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:21.445080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:21.445080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:21.445080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:21.445080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:21.445080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:21.445080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:21.445080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:21.445080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:21.445080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:21.445080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:21.445080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:21.445080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:21.445080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:21.445080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:21.445080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:21.445916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:21.445916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:21.445916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:21.445916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:21.445916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:21.445916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:21.445916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:21.445916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:21.445916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:21.445916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:21.445916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:21.445916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:21.448565 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 193/395] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:21.449520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2f8e5-0d3b-422c-9003-be0e1ffe5f90 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:21.458537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 203/396] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:05:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:21.459595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f2f13-0083-4031-bba1-dc58a2918784 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:21.575765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2f8e5-0d3b-422c-9003-be0e1ffe5f90 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:21.579769 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f2f13-0083-4031-bba1-dc58a2918784 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:21.587537 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:21.602730 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:21.619630 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacc52a-31be-4fc3-b146-b86ae5b0a2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:21.620492 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacc52a-31be-4fc3-b146-b86ae5b0a2ec None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:21.621151 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacc52a-31be-4fc3-b146-b86ae5b0a2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:21.621839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacc52a-31be-4fc3-b146-b86ae5b0a2ec None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:21.623540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacc52a-31be-4fc3-b146-b86ae5b0a2ec None admin] RBAC: Adding query filter params (name=member) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:21.624172 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacc52a-31be-4fc3-b146-b86ae5b0a2ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:21.625658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacc52a-31be-4fc3-b146-b86ae5b0a2ec None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:21.627361 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3205f47-23c7-43ec-8df7-0d0d1eaef6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:21.628909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3205f47-23c7-43ec-8df7-0d0d1eaef6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:21.629439 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3205f47-23c7-43ec-8df7-0d0d1eaef6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:21.629881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3205f47-23c7-43ec-8df7-0d0d1eaef6c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:21.637142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 204/397] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:05:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:21.638090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacc52a-31be-4fc3-b146-b86ae5b0a2ec None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:21.644478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3205f47-23c7-43ec-8df7-0d0d1eaef6c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:21.649879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-d3205f47-23c7-43ec-8df7-0d0d1eaef6c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:05:21.649879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:21.649879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:21.649879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:21.649879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:21.649879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:21.649879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:21.649879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:21.649879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:21.649879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:21.649879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:21.649879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:21.649879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:21.649879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:21.649879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:21.649879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:21.650739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:21.650739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:21.650739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:21.650739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:21.650739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:21.650739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:21.650739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:21.650739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:21.650739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:21.650739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:21.650739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:21.650739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:21.650739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:21.650739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:21.650739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:21.650739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:21.651737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:21.651737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:21.651737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:21.651737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:21.651737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:21.651737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:21.651737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:21.651737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:21.651737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:21.651737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:21.651737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:05:21.651737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:21.653955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 194/398] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:05:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:21.654959 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3205f47-23c7-43ec-8df7-0d0d1eaef6c6 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:21.767596 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacc52a-31be-4fc3-b146-b86ae5b0a2ec None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:21.774401 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3205f47-23c7-43ec-8df7-0d0d1eaef6c6 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:21.806218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bcd8c-06fd-4998-8092-fb76f69c1136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:21.807781 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bcd8c-06fd-4998-8092-fb76f69c1136 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:21.808256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bcd8c-06fd-4998-8092-fb76f69c1136 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:21.808736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bcd8c-06fd-4998-8092-fb76f69c1136 None admin] PATH_INFO: `/v3/groups/366c65011d6f4485b7fac730fd7c4df6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:21.810139 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3bcd8c-06fd-4998-8092-fb76f69c1136 None admin] RBAC: Authorizing `identity:get_group(group_id=366c65011d6f4485b7fac730fd7c4df6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:21.812531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3bcd8c-06fd-4998-8092-fb76f69c1136 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:21.814704 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 205/399] 10.222.0.78 () {56 vars in 1180 bytes} [Mon Sep 19 11:05:21 2022] GET /identity/v3/groups/366c65011d6f4485b7fac730fd7c4df6 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:21.819341 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bcd8c-06fd-4998-8092-fb76f69c1136 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:21.821114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997363b-5fdc-436e-8c8e-f913475610b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:21.821762 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997363b-5fdc-436e-8c8e-f913475610b3 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:21.822613 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997363b-5fdc-436e-8c8e-f913475610b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:21.822891 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997363b-5fdc-436e-8c8e-f913475610b3 None admin] PATH_INFO: `/v3/groups/366c65011d6f4485b7fac730fd7c4df6` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:21.825346 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997363b-5fdc-436e-8c8e-f913475610b3 None admin] RBAC: Authorizing `identity:get_group(group_id=366c65011d6f4485b7fac730fd7c4df6)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:21.826817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997363b-5fdc-436e-8c8e-f913475610b3 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:21.828576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 195/400] 10.222.0.78 () {56 vars in 1180 bytes} [Mon Sep 19 11:05:21 2022] GET /identity/v3/groups/366c65011d6f4485b7fac730fd7c4df6 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:21.829334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997363b-5fdc-436e-8c8e-f913475610b3 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:21.873335 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:21.876093 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:21.927880 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bcd8c-06fd-4998-8092-fb76f69c1136 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:21.940193 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997363b-5fdc-436e-8c8e-f913475610b3 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:21.962640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91d084-eccd-4c1e-8972-52d4ea4165dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:21.963299 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91d084-eccd-4c1e-8972-52d4ea4165dd None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:21.963562 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91d084-eccd-4c1e-8972-52d4ea4165dd None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:21.963837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91d084-eccd-4c1e-8972-52d4ea4165dd None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:21.964497 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b91d084-eccd-4c1e-8972-52d4ea4165dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:21.964809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b91d084-eccd-4c1e-8972-52d4ea4165dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:21.966177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b91d084-eccd-4c1e-8972-52d4ea4165dd None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:21.973719 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72a9bc-f3ae-4b87-809e-65bbb681bac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:21.979623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 206/401] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:21.979767 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72a9bc-f3ae-4b87-809e-65bbb681bac5 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:21.979767 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72a9bc-f3ae-4b87-809e-65bbb681bac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:21.979767 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72a9bc-f3ae-4b87-809e-65bbb681bac5 None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:21.979969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a9f7b-7aec-42a6-8a60-34f221666b86 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:21.979969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a9f7b-7aec-42a6-8a60-34f221666b86 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:21.979969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a9f7b-7aec-42a6-8a60-34f221666b86 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:21.979969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 207/402] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:21.979969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a9f7b-7aec-42a6-8a60-34f221666b86 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:21.981019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72a9bc-f3ae-4b87-809e-65bbb681bac5 None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:21.982476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72a9bc-f3ae-4b87-809e-65bbb681bac5 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:21.984429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 196/403] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:21 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:21.987214 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72a9bc-f3ae-4b87-809e-65bbb681bac5 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:22.093467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72a9bc-f3ae-4b87-809e-65bbb681bac5 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:22.098431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a9f7b-7aec-42a6-8a60-34f221666b86 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:22.100612 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:22.117464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:22.134598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b06b2-25c9-4c42-88b6-58814396da19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:22.135208 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b06b2-25c9-4c42-88b6-58814396da19 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:22.135471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b06b2-25c9-4c42-88b6-58814396da19 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:22.135744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b06b2-25c9-4c42-88b6-58814396da19 None admin] PATH_INFO: `/v3/groups/366c65011d6f4485b7fac730fd7c4df6` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:22.136924 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b06b2-25c9-4c42-88b6-58814396da19 None admin] RBAC: Authorizing `identity:get_group(group_id=366c65011d6f4485b7fac730fd7c4df6)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:22.138308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b06b2-25c9-4c42-88b6-58814396da19 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:22.144618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 197/404] 10.222.0.78 () {56 vars in 1180 bytes} [Mon Sep 19 11:05:21 2022] GET /identity/v3/groups/366c65011d6f4485b7fac730fd7c4df6 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:22.144618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26e469-5429-4571-8757-4151a30ff3ce None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:22.144618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26e469-5429-4571-8757-4151a30ff3ce None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:22.144618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26e469-5429-4571-8757-4151a30ff3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:22.145425 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1d455-d372-45a4-8d5f-63705392b8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:22.146071 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1d455-d372-45a4-8d5f-63705392b8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:22.146410 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1d455-d372-45a4-8d5f-63705392b8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:22.146692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1d455-d372-45a4-8d5f-63705392b8b3 None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:22.148699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1d455-d372-45a4-8d5f-63705392b8b3 None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:22.151805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1d455-d372-45a4-8d5f-63705392b8b3 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:22.153731 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 208/405] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:21 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:22.154280 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1d455-d372-45a4-8d5f-63705392b8b3 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:22.165350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-0a26e469-5429-4571-8757-4151a30ff3ce None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:22.167403 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-0a26e469-5429-4571-8757-4151a30ff3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:22.239701 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 198/406] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:22.240873 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26e469-5429-4571-8757-4151a30ff3ce None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:22.274795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1d455-d372-45a4-8d5f-63705392b8b3 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:22.314443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d108941-19a3-460c-9340-d93050f71672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:22.316033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d108941-19a3-460c-9340-d93050f71672 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:22.316491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d108941-19a3-460c-9340-d93050f71672 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:22.317179 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d108941-19a3-460c-9340-d93050f71672 None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1/groups/366c65011d6f4485b7fac730fd7c4df6/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:22.319517 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d108941-19a3-460c-9340-d93050f71672 None admin] RBAC: Authorizing `identity:create_grant(project_id=1453ab88d2354019a189c0e0883fd8b1, group_id=366c65011d6f4485b7fac730fd7c4df6, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:22.321109 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d108941-19a3-460c-9340-d93050f71672 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:22.359292 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26e469-5429-4571-8757-4151a30ff3ce None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:22.361632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:22.393635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199a42b-81b8-46a5-8c15-f2348d8c89c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:22.394198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199a42b-81b8-46a5-8c15-f2348d8c89c6 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:22.394475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199a42b-81b8-46a5-8c15-f2348d8c89c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:22.394757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199a42b-81b8-46a5-8c15-f2348d8c89c6 None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:22.395922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199a42b-81b8-46a5-8c15-f2348d8c89c6 None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:22.397382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199a42b-81b8-46a5-8c15-f2348d8c89c6 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:22.404661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 199/407] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:22 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:22.406368 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199a42b-81b8-46a5-8c15-f2348d8c89c6 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:22.408877 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 209/408] 10.222.0.78 () {58 vars in 1442 bytes} [Mon Sep 19 11:05:22 2022] PUT /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1/groups/366c65011d6f4485b7fac730fd7c4df6/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:05:22.415457 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d108941-19a3-460c-9340-d93050f71672 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:22.511573 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:22.588535 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199a42b-81b8-46a5-8c15-f2348d8c89c6 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:22.611002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d108941-19a3-460c-9340-d93050f71672 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:22.640344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3668e-2969-438b-8076-d09a5d5736db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:22.647900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3668e-2969-438b-8076-d09a5d5736db None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:22.647900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3668e-2969-438b-8076-d09a5d5736db None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:22.647900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3668e-2969-438b-8076-d09a5d5736db None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780/groups/366c65011d6f4485b7fac730fd7c4df6/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:22.647900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd3668e-2969-438b-8076-d09a5d5736db None admin] RBAC: Authorizing `identity:create_grant(project_id=ff6a8ae52b074b64be28a144288b4780, group_id=366c65011d6f4485b7fac730fd7c4df6, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:22.647900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd3668e-2969-438b-8076-d09a5d5736db None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:22.656930 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061b939-8b38-491d-87ee-96d4796ccd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:22.657582 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061b939-8b38-491d-87ee-96d4796ccd72 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:22.657925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061b939-8b38-491d-87ee-96d4796ccd72 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:22.658204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061b939-8b38-491d-87ee-96d4796ccd72 None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1/groups/366c65011d6f4485b7fac730fd7c4df6/roles/b7eccd60e28d4b39b54879dad8230e0d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:22.660839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061b939-8b38-491d-87ee-96d4796ccd72 None admin] RBAC: Authorizing `identity:create_grant(project_id=1453ab88d2354019a189c0e0883fd8b1, group_id=366c65011d6f4485b7fac730fd7c4df6, role_id=b7eccd60e28d4b39b54879dad8230e0d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:22.661896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061b939-8b38-491d-87ee-96d4796ccd72 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:22.766046 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 200/409] 10.222.0.78 () {58 vars in 1442 bytes} [Mon Sep 19 11:05:22 2022] PUT /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780/groups/366c65011d6f4485b7fac730fd7c4df6/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:05:22.766970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3668e-2969-438b-8076-d09a5d5736db None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:22.806542 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 210/410] 10.222.0.78 () {58 vars in 1442 bytes} [Mon Sep 19 11:05:22 2022] PUT /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1/groups/366c65011d6f4485b7fac730fd7c4df6/roles/b7eccd60e28d4b39b54879dad8230e0d => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:05:22.812600 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e6bb0-2b09-4530-8ff4-d739cf588bde None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:22.814702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e6bb0-2b09-4530-8ff4-d739cf588bde None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:22.814702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e6bb0-2b09-4530-8ff4-d739cf588bde None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:22.815961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 211/411] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:22.818598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:22.820393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2c5ff-f245-4f85-9327-0366951bcf0d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:22.820677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2c5ff-f245-4f85-9327-0366951bcf0d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:22.820959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2c5ff-f245-4f85-9327-0366951bcf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:22.862657 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-66a2c5ff-f245-4f85-9327-0366951bcf0d None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:22.864054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-66a2c5ff-f245-4f85-9327-0366951bcf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:22.924204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3668e-2969-438b-8076-d09a5d5736db None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:22.974885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c811c-0bea-45e0-8356-ab5d81d78533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:22.976174 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c811c-0bea-45e0-8356-ab5d81d78533 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:22.977180 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c811c-0bea-45e0-8356-ab5d81d78533 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:22.977661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c811c-0bea-45e0-8356-ab5d81d78533 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:22.990542 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c811c-0bea-45e0-8356-ab5d81d78533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:22.997885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-504c811c-0bea-45e0-8356-ab5d81d78533 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:05:22.997885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:22.997885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:22.997885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:22.997885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:22.997885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:22.997885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:22.997885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:22.997885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:22.997885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:22.997885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:22.997885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:22.997885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:22.997885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:22.997885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:22.997885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:22.998817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:22.998817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:22.998817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:22.998817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:22.998817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:22.998817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:22.998817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:22.998817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:22.998817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:22.998817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:22.998817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:22.998817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:22.998817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:22.998817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:22.998817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:22.998817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:22.999654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:22.999654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:22.999654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:22.999654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:22.999654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:22.999654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:22.999654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:22.999654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:22.999654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:22.999654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:22.999654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:05:22.999654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:23.002427 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 201/412] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:05:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:23.007339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 212/413] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:23.009309 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c811c-0bea-45e0-8356-ab5d81d78533 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:23.127763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:23.137676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c811c-0bea-45e0-8356-ab5d81d78533 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:23.175059 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608d437-a91d-4b47-aae4-f514e8097762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:23.175819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608d437-a91d-4b47-aae4-f514e8097762 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:23.176118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608d437-a91d-4b47-aae4-f514e8097762 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:23.176389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608d437-a91d-4b47-aae4-f514e8097762 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:23.177121 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608d437-a91d-4b47-aae4-f514e8097762 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:23.177425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608d437-a91d-4b47-aae4-f514e8097762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:23.178850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608d437-a91d-4b47-aae4-f514e8097762 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:23.188539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 202/414] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:23.194267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2c5ff-f245-4f85-9327-0366951bcf0d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:23.249727 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:23.370084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2c5ff-f245-4f85-9327-0366951bcf0d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:23.416601 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606164a-3a66-491c-b71a-785581384240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:23.417259 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606164a-3a66-491c-b71a-785581384240 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:23.417540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606164a-3a66-491c-b71a-785581384240 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:23.417813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606164a-3a66-491c-b71a-785581384240 None admin] PATH_INFO: `/v3/groups/0335f2cbdb24418ebf1b1a26dae9f7e5` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:23.418956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606164a-3a66-491c-b71a-785581384240 None admin] RBAC: Authorizing `identity:get_group(group_id=0335f2cbdb24418ebf1b1a26dae9f7e5)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:23.420438 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606164a-3a66-491c-b71a-785581384240 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:23.422372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 213/415] 10.222.0.78 () {56 vars in 1180 bytes} [Mon Sep 19 11:05:23 2022] GET /identity/v3/groups/0335f2cbdb24418ebf1b1a26dae9f7e5 => generated 229 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:23.427955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608d437-a91d-4b47-aae4-f514e8097762 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:23.471023 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606164a-3a66-491c-b71a-785581384240 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:23.557946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608d437-a91d-4b47-aae4-f514e8097762 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:23.597421 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e5998-bf2d-4367-a2f2-0804cb39bb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:23.598126 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e5998-bf2d-4367-a2f2-0804cb39bb73 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:23.598407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e5998-bf2d-4367-a2f2-0804cb39bb73 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:23.598732 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e5998-bf2d-4367-a2f2-0804cb39bb73 None admin] PATH_INFO: `/v3/projects/a875b13c6f2b4a43a97ce59e85aa99fd` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:23.600079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e5998-bf2d-4367-a2f2-0804cb39bb73 None admin] RBAC: Authorizing `identity:get_project(project_id=a875b13c6f2b4a43a97ce59e85aa99fd)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:23.602005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e5998-bf2d-4367-a2f2-0804cb39bb73 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:23.609781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 203/416] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:23 2022] GET /identity/v3/projects/a875b13c6f2b4a43a97ce59e85aa99fd => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:23.611231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e5998-bf2d-4367-a2f2-0804cb39bb73 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:23.648363 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:23.687842 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:23.693750 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606164a-3a66-491c-b71a-785581384240 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:23.761657 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfa767-4fd4-4de5-9640-32d4cd9052e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:23.762360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfa767-4fd4-4de5-9640-32d4cd9052e3 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:23.762642 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfa767-4fd4-4de5-9640-32d4cd9052e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:23.762915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfa767-4fd4-4de5-9640-32d4cd9052e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:23.784237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfa767-4fd4-4de5-9640-32d4cd9052e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:23.790030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-f3bfa767-4fd4-4de5-9640-32d4cd9052e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:05:23.790030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:23.790030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:23.790030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:23.790030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:23.790030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:23.790030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:23.790030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:23.790030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:23.790030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:23.790030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:23.790030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:23.790030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:23.790030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:23.790030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:23.790030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:23.791632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:23.791632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:23.791632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:23.791632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:23.791632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:23.791632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:23.791632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:23.791632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:23.791632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:23.791632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:23.791632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:23.791632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:23.791632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:23.791632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:23.791632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:23.791632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:23.792415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:23.792415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:23.792415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:23.792415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:23.792415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:23.792415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:23.792415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:23.792415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:23.792415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:23.792415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:23.792415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:05:23.792415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:23.799710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e5998-bf2d-4367-a2f2-0804cb39bb73 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:23.801653 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 214/417] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:05:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:23.816749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfa767-4fd4-4de5-9640-32d4cd9052e3 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:23.839866 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd07570-24ce-4424-852e-6308c9b1204c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:23.840544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd07570-24ce-4424-852e-6308c9b1204c None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:23.840884 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd07570-24ce-4424-852e-6308c9b1204c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:23.841168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd07570-24ce-4424-852e-6308c9b1204c None admin] PATH_INFO: `/v3/projects/a875b13c6f2b4a43a97ce59e85aa99fd/groups/0335f2cbdb24418ebf1b1a26dae9f7e5/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:23.844265 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd07570-24ce-4424-852e-6308c9b1204c None admin] RBAC: Authorizing `identity:create_grant(project_id=a875b13c6f2b4a43a97ce59e85aa99fd, group_id=0335f2cbdb24418ebf1b1a26dae9f7e5, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:23.849421 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd07570-24ce-4424-852e-6308c9b1204c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:23.940870 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfa767-4fd4-4de5-9640-32d4cd9052e3 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:23.941688 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 204/418] 10.222.0.78 () {58 vars in 1442 bytes} [Mon Sep 19 11:05:23 2022] PUT /identity/v3/projects/a875b13c6f2b4a43a97ce59e85aa99fd/groups/0335f2cbdb24418ebf1b1a26dae9f7e5/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:05:23.992831 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c7ffb-10da-4106-a653-9d103f392f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:23.993476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c7ffb-10da-4106-a653-9d103f392f7d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:23.993853 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c7ffb-10da-4106-a653-9d103f392f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:23.994134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c7ffb-10da-4106-a653-9d103f392f7d None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:23.994837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c7ffb-10da-4106-a653-9d103f392f7d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:23.995288 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c7ffb-10da-4106-a653-9d103f392f7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:23.996710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c7ffb-10da-4106-a653-9d103f392f7d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:24.011368 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 215/419] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:24.012968 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd07570-24ce-4424-852e-6308c9b1204c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:24.143509 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:24.168885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd07570-24ce-4424-852e-6308c9b1204c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:24.206411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6697c-0da6-4854-9736-9154bbe899fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:24.206954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6697c-0da6-4854-9736-9154bbe899fe None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:24.207262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6697c-0da6-4854-9736-9154bbe899fe None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:24.207541 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6697c-0da6-4854-9736-9154bbe899fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:24.280754 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf6697c-0da6-4854-9736-9154bbe899fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:24.282305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf6697c-0da6-4854-9736-9154bbe899fe None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:24.379996 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:24.400885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 205/420] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 11:05:24.407077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c7ffb-10da-4106-a653-9d103f392f7d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:24.432849 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:24.515486 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c7ffb-10da-4106-a653-9d103f392f7d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:24.549985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afbbba-54b5-4a43-af1e-c80a8ea867f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:24.550618 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afbbba-54b5-4a43-af1e-c80a8ea867f1 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:24.550901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afbbba-54b5-4a43-af1e-c80a8ea867f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:24.551199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afbbba-54b5-4a43-af1e-c80a8ea867f1 None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:24.552337 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36afbbba-54b5-4a43-af1e-c80a8ea867f1 None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:24.553768 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36afbbba-54b5-4a43-af1e-c80a8ea867f1 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:24.560529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 216/421] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:24 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:24.561911 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6697c-0da6-4854-9736-9154bbe899fe None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:24.674788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6697c-0da6-4854-9736-9154bbe899fe None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:24.722929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8d1ba-796d-4669-ac7f-e5af519bc18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:24.723613 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8d1ba-796d-4669-ac7f-e5af519bc18a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:24.723897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8d1ba-796d-4669-ac7f-e5af519bc18a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:24.724169 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8d1ba-796d-4669-ac7f-e5af519bc18a None admin] PATH_INFO: `/v3/groups/366c65011d6f4485b7fac730fd7c4df6` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:24.725303 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8d1ba-796d-4669-ac7f-e5af519bc18a None admin] RBAC: Authorizing `identity:get_group(group_id=366c65011d6f4485b7fac730fd7c4df6)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:24.726761 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8d1ba-796d-4669-ac7f-e5af519bc18a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:24.728953 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 206/422] 10.222.0.78 () {56 vars in 1180 bytes} [Mon Sep 19 11:05:24 2022] GET /identity/v3/groups/366c65011d6f4485b7fac730fd7c4df6 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:24.735084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afbbba-54b5-4a43-af1e-c80a8ea867f1 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:24.835620 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:24.843097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afbbba-54b5-4a43-af1e-c80a8ea867f1 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:24.878687 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec916e10-3ea8-48c4-ab03-712f3a1b906b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:24.879335 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec916e10-3ea8-48c4-ab03-712f3a1b906b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:24.879667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec916e10-3ea8-48c4-ab03-712f3a1b906b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:24.879953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec916e10-3ea8-48c4-ab03-712f3a1b906b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:24.880700 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec916e10-3ea8-48c4-ab03-712f3a1b906b None admin] RBAC: Adding query filter params (group.id=366c65011d6f4485b7fac730fd7c4df6, role.id=b7eccd60e28d4b39b54879dad8230e0d, scope.project.id=ff6a8ae52b074b64be28a144288b4780) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:24.881010 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec916e10-3ea8-48c4-ab03-712f3a1b906b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:24.882440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec916e10-3ea8-48c4-ab03-712f3a1b906b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:24.889482 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 217/423] 10.222.0.78 () {56 vars in 1376 bytes} [Mon Sep 19 11:05:24 2022] GET /identity/v3/role_assignments?group.id=366c65011d6f4485b7fac730fd7c4df6&scope.project.id=ff6a8ae52b074b64be28a144288b4780&role.id=b7eccd60e28d4b39b54879dad8230e0d => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:25.207885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37c2f9-bc40-429d-b565-b230c7c8fef9 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:25.208329 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37c2f9-bc40-429d-b565-b230c7c8fef9 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:25.208710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37c2f9-bc40-429d-b565-b230c7c8fef9 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:25.210699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 207/424] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:25.216266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba168fa-ff3f-4402-9734-b221e5386619 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:25.216615 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba168fa-ff3f-4402-9734-b221e5386619 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:25.216827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba168fa-ff3f-4402-9734-b221e5386619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:25.242454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-3ba168fa-ff3f-4402-9734-b221e5386619 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:25.244249 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3ba168fa-ff3f-4402-9734-b221e5386619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:25.338994 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 218/425] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:25.359243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557515b4-fb85-4853-b237-0ffc120cec9d None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:25.359584 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557515b4-fb85-4853-b237-0ffc120cec9d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:25.359797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557515b4-fb85-4853-b237-0ffc120cec9d None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:25.361536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 208/426] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:25.367426 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101eaa7d-c4e9-4268-b05b-aa9f806e904a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:25.367704 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101eaa7d-c4e9-4268-b05b-aa9f806e904a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:25.367896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101eaa7d-c4e9-4268-b05b-aa9f806e904a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:25.400372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-101eaa7d-c4e9-4268-b05b-aa9f806e904a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:25.401973 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-101eaa7d-c4e9-4268-b05b-aa9f806e904a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:25.473277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 219/427] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:25.491285 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25bbad-bea7-4ab8-8ff2-91cf1f135ece None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:25.491568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25bbad-bea7-4ab8-8ff2-91cf1f135ece None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:25.491849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25bbad-bea7-4ab8-8ff2-91cf1f135ece None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:25.493470 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 209/428] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:25.500634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d8b8a-ba9c-4749-8011-e73ab7249ce0 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:25.500922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d8b8a-ba9c-4749-8011-e73ab7249ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:25.501198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d8b8a-ba9c-4749-8011-e73ab7249ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:25.520516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ce9d8b8a-ba9c-4749-8011-e73ab7249ce0 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:25.521714 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ce9d8b8a-ba9c-4749-8011-e73ab7249ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:25.529429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:25.590756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 220/429] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:25.631728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25bbad-bea7-4ab8-8ff2-91cf1f135ece None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:25.686021 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:25.739332 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25bbad-bea7-4ab8-8ff2-91cf1f135ece None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:25.760235 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d8b8a-ba9c-4749-8011-e73ab7249ce0 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:25.779278 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae14cf-8ec7-4e4d-8913-40dddec8e136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:25.779926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae14cf-8ec7-4e4d-8913-40dddec8e136 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:25.780205 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae14cf-8ec7-4e4d-8913-40dddec8e136 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:25.780478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae14cf-8ec7-4e4d-8913-40dddec8e136 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:25.791803 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae14cf-8ec7-4e4d-8913-40dddec8e136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:25.797434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-0fae14cf-8ec7-4e4d-8913-40dddec8e136 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:25.797434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:25.797434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:25.797434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:25.797434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:25.797434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:25.797434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:25.797434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:25.797434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:25.797434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:25.797434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:25.797434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:25.797434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:25.797434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:25.797434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:25.797434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:25.798524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:25.798524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:25.798524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:25.798524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:25.798524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:25.798524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:25.798524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:25.798524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:25.798524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:25.798524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:25.798524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:25.798524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:25.798524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:25.798524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:25.798524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:25.798524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:25.799533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:25.799533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:25.799533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:25.799533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:25.799533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:25.799533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:25.799533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:25.799533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:25.799533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:25.799533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:25.799533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:25.799533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:25.799533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 210/430] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:25.806168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae14cf-8ec7-4e4d-8913-40dddec8e136 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:25.880499 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d8b8a-ba9c-4749-8011-e73ab7249ce0 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:25.913666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae14cf-8ec7-4e4d-8913-40dddec8e136 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:25.915417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825eca95-0220-47e5-8b72-b8b52396246e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:25.916593 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825eca95-0220-47e5-8b72-b8b52396246e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:25.916875 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825eca95-0220-47e5-8b72-b8b52396246e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:25.917145 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825eca95-0220-47e5-8b72-b8b52396246e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:25.926447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:25.928725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825eca95-0220-47e5-8b72-b8b52396246e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:25.934202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-825eca95-0220-47e5-8b72-b8b52396246e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:25.934202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:25.934202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:25.934202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:25.934202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:25.934202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:25.934202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:25.934202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:25.934202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:25.934202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:25.934202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:25.934202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:25.934202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:25.934202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:25.934202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:25.934202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:25.940699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:25.940699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:25.940699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:25.940699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:25.940699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:25.940699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:25.940699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:25.940699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:25.940699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:25.940699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:25.940699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:25.940699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:25.940699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:25.940699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:25.940699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:25.940699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:25.941506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:25.941506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:25.941506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:25.941506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:25.941506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:25.941506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:25.941506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:25.941506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:25.941506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:25.941506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:25.941506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:05:25.941506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:25.941506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 221/431] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:25.941506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825eca95-0220-47e5-8b72-b8b52396246e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:25.943553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:25.960482 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe50157-576d-4736-9d2e-80aa7cd58e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:25.961402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe50157-576d-4736-9d2e-80aa7cd58e31 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:25.961827 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe50157-576d-4736-9d2e-80aa7cd58e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:25.962244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe50157-576d-4736-9d2e-80aa7cd58e31 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:25.963634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe50157-576d-4736-9d2e-80aa7cd58e31 None admin] RBAC: Adding query filter params (name=member) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:25.964084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe50157-576d-4736-9d2e-80aa7cd58e31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:25.971771 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe50157-576d-4736-9d2e-80aa7cd58e31 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:25.979867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 211/432] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:05:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:25.980278 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe50157-576d-4736-9d2e-80aa7cd58e31 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:26.053771 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825eca95-0220-47e5-8b72-b8b52396246e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:26.085506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe50157-576d-4736-9d2e-80aa7cd58e31 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:26.096276 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c6661-08e2-4511-8228-356ffb86ff8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:26.097340 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c6661-08e2-4511-8228-356ffb86ff8a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:26.097764 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c6661-08e2-4511-8228-356ffb86ff8a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:26.098041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c6661-08e2-4511-8228-356ffb86ff8a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:26.110245 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c6661-08e2-4511-8228-356ffb86ff8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:26.115972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-b46c6661-08e2-4511-8228-356ffb86ff8a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:05:26.115972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:26.115972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:26.115972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:26.115972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:26.115972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:26.115972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:26.115972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:26.115972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:26.115972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:26.115972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:26.115972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:26.115972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:26.115972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:26.115972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:26.115972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:26.116850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:26.116850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:26.116850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:26.116850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:26.116850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:26.116850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:26.116850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:26.116850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:26.116850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:26.116850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:26.116850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:26.116850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:26.116850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:26.116850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:26.116850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:26.116850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:26.117670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:26.117670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:26.117670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:26.117670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:26.117670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:26.117670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:26.117670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:26.117670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:26.117670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:26.117670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:26.117670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:05:26.117670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:26.121989 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 222/433] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:05:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:26.122695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c6661-08e2-4511-8228-356ffb86ff8a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:26.125221 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293284df-b49f-45e4-bbba-3958af85edf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:26.126118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293284df-b49f-45e4-bbba-3958af85edf4 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:26.126403 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293284df-b49f-45e4-bbba-3958af85edf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:26.127318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293284df-b49f-45e4-bbba-3958af85edf4 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:26.128288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293284df-b49f-45e4-bbba-3958af85edf4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:26.128599 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293284df-b49f-45e4-bbba-3958af85edf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:26.129974 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293284df-b49f-45e4-bbba-3958af85edf4 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:26.144192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 212/434] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:05:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:26.144192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293284df-b49f-45e4-bbba-3958af85edf4 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:26.177161 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:26.198668 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:26.241848 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c6661-08e2-4511-8228-356ffb86ff8a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:26.245623 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293284df-b49f-45e4-bbba-3958af85edf4 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:26.279931 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9641d27-b492-4839-b32a-6e12d4e444c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:26.280907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9641d27-b492-4839-b32a-6e12d4e444c9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:26.281390 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9641d27-b492-4839-b32a-6e12d4e444c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:26.281739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3cdfa5-bced-4397-8481-12e5f3f6dce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:26.282138 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9641d27-b492-4839-b32a-6e12d4e444c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:26.282355 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3cdfa5-bced-4397-8481-12e5f3f6dce3 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:26.282637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3cdfa5-bced-4397-8481-12e5f3f6dce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:26.282913 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3cdfa5-bced-4397-8481-12e5f3f6dce3 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:26.283612 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3cdfa5-bced-4397-8481-12e5f3f6dce3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:26.283919 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3cdfa5-bced-4397-8481-12e5f3f6dce3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:26.285294 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3cdfa5-bced-4397-8481-12e5f3f6dce3 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:26.294262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 213/435] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:26.294262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3cdfa5-bced-4397-8481-12e5f3f6dce3 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:26.367960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9641d27-b492-4839-b32a-6e12d4e444c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:26.369466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9641d27-b492-4839-b32a-6e12d4e444c9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:26.408262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:26.414671 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3cdfa5-bced-4397-8481-12e5f3f6dce3 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:26.444801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:26.450765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbf889-28de-49b0-9fde-beb509f56a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:26.451410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbf889-28de-49b0-9fde-beb509f56a9d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:26.451691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbf889-28de-49b0-9fde-beb509f56a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:26.451964 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbf889-28de-49b0-9fde-beb509f56a9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:26.490472 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 223/436] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:26.491677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9641d27-b492-4839-b32a-6e12d4e444c9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:26.530891 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbf889-28de-49b0-9fde-beb509f56a9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:26.533502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbf889-28de-49b0-9fde-beb509f56a9d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:26.609633 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9641d27-b492-4839-b32a-6e12d4e444c9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:26.641090 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:26.648976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 214/437] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:26.649820 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbf889-28de-49b0-9fde-beb509f56a9d None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:26.655915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84f9c9-a5e7-4f41-8468-d15c68148340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:26.656665 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84f9c9-a5e7-4f41-8468-d15c68148340 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:26.656954 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84f9c9-a5e7-4f41-8468-d15c68148340 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:26.657229 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84f9c9-a5e7-4f41-8468-d15c68148340 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:26.692337 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:26.733426 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84f9c9-a5e7-4f41-8468-d15c68148340 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:26.734796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84f9c9-a5e7-4f41-8468-d15c68148340 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:26.752088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbf889-28de-49b0-9fde-beb509f56a9d None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:26.786691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac3275-3c15-4ab9-976b-49ff9182cf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:26.787381 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac3275-3c15-4ab9-976b-49ff9182cf1d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:26.787663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac3275-3c15-4ab9-976b-49ff9182cf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:26.787936 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac3275-3c15-4ab9-976b-49ff9182cf1d None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:26.789068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac3275-3c15-4ab9-976b-49ff9182cf1d None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:26.790526 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac3275-3c15-4ab9-976b-49ff9182cf1d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:26.792347 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 215/438] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:26 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:26.792947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac3275-3c15-4ab9-976b-49ff9182cf1d None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:26.839699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 224/439] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:26.841031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9642e-c845-4233-83d3-d8ed4443ee58 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:26.841320 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9642e-c845-4233-83d3-d8ed4443ee58 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:26.841598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9642e-c845-4233-83d3-d8ed4443ee58 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:26.842817 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 225/440] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:26.843542 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9642e-c845-4233-83d3-d8ed4443ee58 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:26.859918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:26.898555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac3275-3c15-4ab9-976b-49ff9182cf1d None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:26.907647 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:26.937032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b074b2-617c-4d1d-8a15-e897a484105e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:26.937657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b074b2-617c-4d1d-8a15-e897a484105e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:26.937942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b074b2-617c-4d1d-8a15-e897a484105e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:26.938216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b074b2-617c-4d1d-8a15-e897a484105e None admin] PATH_INFO: `/v3/projects/ff6a8ae52b074b64be28a144288b4780` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:26.939404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b074b2-617c-4d1d-8a15-e897a484105e None admin] RBAC: Authorizing `identity:get_project(project_id=ff6a8ae52b074b64be28a144288b4780)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:26.940848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b074b2-617c-4d1d-8a15-e897a484105e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:26.942459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 216/441] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:26 2022] GET /identity/v3/projects/ff6a8ae52b074b64be28a144288b4780 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:26.943237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b074b2-617c-4d1d-8a15-e897a484105e None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:26.950478 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9642e-c845-4233-83d3-d8ed4443ee58 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:26.983075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9c1a3-df8a-473a-b921-8d8dc55d170b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:26.983972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9c1a3-df8a-473a-b921-8d8dc55d170b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:26.984251 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9c1a3-df8a-473a-b921-8d8dc55d170b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:26.984591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9c1a3-df8a-473a-b921-8d8dc55d170b None admin] PATH_INFO: `/v3/groups/366c65011d6f4485b7fac730fd7c4df6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:26.985907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9c1a3-df8a-473a-b921-8d8dc55d170b None admin] RBAC: Authorizing `identity:get_group(group_id=366c65011d6f4485b7fac730fd7c4df6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:26.987728 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9c1a3-df8a-473a-b921-8d8dc55d170b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:26.989420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 226/442] 10.222.0.78 () {56 vars in 1180 bytes} [Mon Sep 19 11:05:26 2022] GET /identity/v3/groups/366c65011d6f4485b7fac730fd7c4df6 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:26.995429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c85e29-373c-4fcb-b6ea-ad917425979a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:26.995710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c85e29-373c-4fcb-b6ea-ad917425979a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:26.995979 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c85e29-373c-4fcb-b6ea-ad917425979a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:27.014194 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a0c85e29-373c-4fcb-b6ea-ad917425979a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:27.015405 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a0c85e29-373c-4fcb-b6ea-ad917425979a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:27.042474 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b074b2-617c-4d1d-8a15-e897a484105e None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:27.072622 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:27.077512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562c199-6017-4e32-8212-220ba4e209d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:27.078215 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562c199-6017-4e32-8212-220ba4e209d9 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:27.078500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562c199-6017-4e32-8212-220ba4e209d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:27.078772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562c199-6017-4e32-8212-220ba4e209d9 None admin] PATH_INFO: `/v3/projects/1453ab88d2354019a189c0e0883fd8b1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:27.079926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562c199-6017-4e32-8212-220ba4e209d9 None admin] RBAC: Authorizing `identity:get_project(project_id=1453ab88d2354019a189c0e0883fd8b1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:27.081399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562c199-6017-4e32-8212-220ba4e209d9 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:27.082727 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 227/443] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:27.083060 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 217/444] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:26 2022] GET /identity/v3/projects/1453ab88d2354019a189c0e0883fd8b1 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:27.083600 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c85e29-373c-4fcb-b6ea-ad917425979a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:27.084954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562c199-6017-4e32-8212-220ba4e209d9 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:27.150166 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:27.197023 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c85e29-373c-4fcb-b6ea-ad917425979a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:27.197216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562c199-6017-4e32-8212-220ba4e209d9 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:27.232812 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422722b5-3b8c-44fa-80ac-03b967a937fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:27.233424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422722b5-3b8c-44fa-80ac-03b967a937fd None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:27.233530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3acb92-2094-4c42-b54f-b207a54ba824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:27.233735 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422722b5-3b8c-44fa-80ac-03b967a937fd None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:27.234089 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422722b5-3b8c-44fa-80ac-03b967a937fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:27.234204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3acb92-2094-4c42-b54f-b207a54ba824 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:27.234374 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3acb92-2094-4c42-b54f-b207a54ba824 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:27.234644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3acb92-2094-4c42-b54f-b207a54ba824 None admin] PATH_INFO: `/v3/groups/366c65011d6f4485b7fac730fd7c4df6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:27.234798 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422722b5-3b8c-44fa-80ac-03b967a937fd None admin] RBAC: Adding query filter params (group.id=366c65011d6f4485b7fac730fd7c4df6, role.id=486614b0ab4b454b92e9ef0bc7ec61c1, scope.project.id=1453ab88d2354019a189c0e0883fd8b1) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:27.235065 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422722b5-3b8c-44fa-80ac-03b967a937fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:27.235881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3acb92-2094-4c42-b54f-b207a54ba824 None admin] RBAC: Authorizing `identity:get_group(group_id=366c65011d6f4485b7fac730fd7c4df6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:27.236532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422722b5-3b8c-44fa-80ac-03b967a937fd None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:27.237289 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3acb92-2094-4c42-b54f-b207a54ba824 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:27.239090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 228/445] 10.222.0.78 () {56 vars in 1180 bytes} [Mon Sep 19 11:05:27 2022] GET /identity/v3/groups/366c65011d6f4485b7fac730fd7c4df6 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:27.251640 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 218/446] 10.222.0.78 () {56 vars in 1376 bytes} [Mon Sep 19 11:05:27 2022] GET /identity/v3/role_assignments?group.id=366c65011d6f4485b7fac730fd7c4df6&scope.project.id=1453ab88d2354019a189c0e0883fd8b1&role.id=486614b0ab4b454b92e9ef0bc7ec61c1 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:27.251864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3acb92-2094-4c42-b54f-b207a54ba824 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:27.251943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422722b5-3b8c-44fa-80ac-03b967a937fd None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:27.326645 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:27.371041 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422722b5-3b8c-44fa-80ac-03b967a937fd None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:27.383839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3acb92-2094-4c42-b54f-b207a54ba824 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:27.410553 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec721fed-fc0c-4660-bf9b-e524f0baf590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:27.411341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec721fed-fc0c-4660-bf9b-e524f0baf590 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:27.411694 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec721fed-fc0c-4660-bf9b-e524f0baf590 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:27.412038 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec721fed-fc0c-4660-bf9b-e524f0baf590 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:27.413078 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec721fed-fc0c-4660-bf9b-e524f0baf590 None admin] RBAC: Adding query filter params (group.id=366c65011d6f4485b7fac730fd7c4df6, role.id=486614b0ab4b454b92e9ef0bc7ec61c1, scope.project.id=ff6a8ae52b074b64be28a144288b4780) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:27.413199 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec721fed-fc0c-4660-bf9b-e524f0baf590 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:27.414687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec721fed-fc0c-4660-bf9b-e524f0baf590 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:27.423933 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 219/447] 10.222.0.78 () {56 vars in 1376 bytes} [Mon Sep 19 11:05:27 2022] GET /identity/v3/role_assignments?group.id=366c65011d6f4485b7fac730fd7c4df6&scope.project.id=ff6a8ae52b074b64be28a144288b4780&role.id=486614b0ab4b454b92e9ef0bc7ec61c1 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:27.424629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec721fed-fc0c-4660-bf9b-e524f0baf590 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:27.434072 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:27.444738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89254f0-fe66-46f6-8032-6bee3c0beac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:27.446062 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89254f0-fe66-46f6-8032-6bee3c0beac2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:27.446062 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89254f0-fe66-46f6-8032-6bee3c0beac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:27.447344 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89254f0-fe66-46f6-8032-6bee3c0beac2 None admin] PATH_INFO: `/v3/groups/366c65011d6f4485b7fac730fd7c4df6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:27.448201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89254f0-fe66-46f6-8032-6bee3c0beac2 None admin] RBAC: Authorizing `identity:get_group(group_id=366c65011d6f4485b7fac730fd7c4df6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:27.449491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89254f0-fe66-46f6-8032-6bee3c0beac2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:27.452201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 229/448] 10.222.0.78 () {56 vars in 1180 bytes} [Mon Sep 19 11:05:27 2022] GET /identity/v3/groups/366c65011d6f4485b7fac730fd7c4df6 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:27.458272 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89254f0-fe66-46f6-8032-6bee3c0beac2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:27.564535 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec721fed-fc0c-4660-bf9b-e524f0baf590 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:27.595493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89254f0-fe66-46f6-8032-6bee3c0beac2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:27.596247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:27.606371 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd866349-2781-412a-80b5-553cf49d0f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:27.607422 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd866349-2781-412a-80b5-553cf49d0f05 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:27.607707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd866349-2781-412a-80b5-553cf49d0f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:27.607981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd866349-2781-412a-80b5-553cf49d0f05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:27.620672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd866349-2781-412a-80b5-553cf49d0f05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:27.627099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-dd866349-2781-412a-80b5-553cf49d0f05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:05:27.627099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:27.627099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:27.627099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:27.627099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:27.627099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:27.627099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:27.627099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:27.627099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:27.627099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:27.627099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:27.627099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:27.627099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:27.627099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:27.627099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:27.627099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:27.628288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:27.628288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:27.628288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:27.628288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:27.628288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:27.628288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:27.628288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:27.628288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:27.628288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:27.628288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:27.628288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:27.628288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:27.628288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:27.628288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:27.628288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:27.628288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:27.629114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:27.629114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:27.629114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:27.629114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:27.629114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:27.629114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:27.629114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:27.629114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:27.629114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:27.629114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:27.629114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:05:27.629114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:27.629114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 220/449] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:05:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:27.634224 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd866349-2781-412a-80b5-553cf49d0f05 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:27.637031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6efc76-5524-4198-8f15-b87034f1c59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:27.637651 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6efc76-5524-4198-8f15-b87034f1c59f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:27.637937 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6efc76-5524-4198-8f15-b87034f1c59f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:27.638211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6efc76-5524-4198-8f15-b87034f1c59f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:27.638966 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6efc76-5524-4198-8f15-b87034f1c59f None admin] RBAC: Adding query filter params (group.id=366c65011d6f4485b7fac730fd7c4df6, role.id=b7eccd60e28d4b39b54879dad8230e0d, scope.project.id=1453ab88d2354019a189c0e0883fd8b1) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:27.640248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6efc76-5524-4198-8f15-b87034f1c59f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:27.640794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6efc76-5524-4198-8f15-b87034f1c59f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:27.648183 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 230/450] 10.222.0.78 () {56 vars in 1376 bytes} [Mon Sep 19 11:05:27 2022] GET /identity/v3/role_assignments?group.id=366c65011d6f4485b7fac730fd7c4df6&scope.project.id=1453ab88d2354019a189c0e0883fd8b1&role.id=b7eccd60e28d4b39b54879dad8230e0d => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:27.757767 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd866349-2781-412a-80b5-553cf49d0f05 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:27.800517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfee7c6-8a4e-42a2-a54b-c6c462a57d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:27.801186 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfee7c6-8a4e-42a2-a54b-c6c462a57d38 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:27.802072 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfee7c6-8a4e-42a2-a54b-c6c462a57d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:27.802072 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfee7c6-8a4e-42a2-a54b-c6c462a57d38 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:27.802404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfee7c6-8a4e-42a2-a54b-c6c462a57d38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:27.802710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfee7c6-8a4e-42a2-a54b-c6c462a57d38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:27.804209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfee7c6-8a4e-42a2-a54b-c6c462a57d38 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:27.814854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 221/451] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:27.820420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6efc76-5524-4198-8f15-b87034f1c59f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:27.884753 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:27.928041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6efc76-5524-4198-8f15-b87034f1c59f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:27.963043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67b7b0-c269-4a02-8868-30305fca45b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:27.963751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67b7b0-c269-4a02-8868-30305fca45b0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:27.964036 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67b7b0-c269-4a02-8868-30305fca45b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:27.964312 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67b7b0-c269-4a02-8868-30305fca45b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:28.034297 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67b7b0-c269-4a02-8868-30305fca45b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:28.035779 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67b7b0-c269-4a02-8868-30305fca45b0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:28.096792 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:28.139619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 231/452] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:28.145428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfee7c6-8a4e-42a2-a54b-c6c462a57d38 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:28.200546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:28.251555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfee7c6-8a4e-42a2-a54b-c6c462a57d38 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:28.286377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c63523-df7f-42e4-81c2-62d010d6089b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:28.286908 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c63523-df7f-42e4-81c2-62d010d6089b None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:28.287210 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c63523-df7f-42e4-81c2-62d010d6089b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:28.287478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c63523-df7f-42e4-81c2-62d010d6089b None admin] PATH_INFO: `/v3/projects/a875b13c6f2b4a43a97ce59e85aa99fd` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:28.288596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c63523-df7f-42e4-81c2-62d010d6089b None admin] RBAC: Authorizing `identity:get_project(project_id=a875b13c6f2b4a43a97ce59e85aa99fd)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:28.290025 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c63523-df7f-42e4-81c2-62d010d6089b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:28.291706 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 222/453] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:05:28 2022] GET /identity/v3/projects/a875b13c6f2b4a43a97ce59e85aa99fd => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:28.297252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67b7b0-c269-4a02-8868-30305fca45b0 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:28.395734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67b7b0-c269-4a02-8868-30305fca45b0 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:28.428199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38384ca1-dcff-415c-9b2d-4437be9fd8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:28.428834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38384ca1-dcff-415c-9b2d-4437be9fd8ee None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:28.429120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38384ca1-dcff-415c-9b2d-4437be9fd8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:28.429399 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38384ca1-dcff-415c-9b2d-4437be9fd8ee None admin] PATH_INFO: `/v3/groups/0335f2cbdb24418ebf1b1a26dae9f7e5` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:28.430531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38384ca1-dcff-415c-9b2d-4437be9fd8ee None admin] RBAC: Authorizing `identity:get_group(group_id=0335f2cbdb24418ebf1b1a26dae9f7e5)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:28.432552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38384ca1-dcff-415c-9b2d-4437be9fd8ee None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:28.434126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 232/454] 10.222.0.78 () {56 vars in 1180 bytes} [Mon Sep 19 11:05:28 2022] GET /identity/v3/groups/0335f2cbdb24418ebf1b1a26dae9f7e5 => generated 229 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:28.440028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c63523-df7f-42e4-81c2-62d010d6089b None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:28.536588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c63523-df7f-42e4-81c2-62d010d6089b None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:28.552533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:28.568732 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b326e9-1fc7-42dc-a67b-fd98956951bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:28.569324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b326e9-1fc7-42dc-a67b-fd98956951bd None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:28.569602 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b326e9-1fc7-42dc-a67b-fd98956951bd None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:28.569869 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b326e9-1fc7-42dc-a67b-fd98956951bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:28.570578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b326e9-1fc7-42dc-a67b-fd98956951bd None admin] RBAC: Adding query filter params (group.id=0335f2cbdb24418ebf1b1a26dae9f7e5, role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=a875b13c6f2b4a43a97ce59e85aa99fd) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:28.570879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b326e9-1fc7-42dc-a67b-fd98956951bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:28.572947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b326e9-1fc7-42dc-a67b-fd98956951bd None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:28.578467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 223/455] 10.222.0.78 () {56 vars in 1376 bytes} [Mon Sep 19 11:05:28 2022] GET /identity/v3/role_assignments?group.id=0335f2cbdb24418ebf1b1a26dae9f7e5&scope.project.id=a875b13c6f2b4a43a97ce59e85aa99fd&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:31.099592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4004ce1-34ae-4c70-bf13-0d25af3be743 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:31.100161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4004ce1-34ae-4c70-bf13-0d25af3be743 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:31.100161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4004ce1-34ae-4c70-bf13-0d25af3be743 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:31.101750 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 233/456] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:31.115379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5ec06-fe16-4ad3-9eac-7fed10e06df2 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:31.115791 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5ec06-fe16-4ad3-9eac-7fed10e06df2 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:31.115953 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5ec06-fe16-4ad3-9eac-7fed10e06df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:31.142977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-2fe5ec06-fe16-4ad3-9eac-7fed10e06df2 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:31.144372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2fe5ec06-fe16-4ad3-9eac-7fed10e06df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:31.231876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 224/457] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:31.641204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3913c-cc97-4947-b1d9-3406bed6ba3e None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:31.641605 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3913c-cc97-4947-b1d9-3406bed6ba3e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:31.641787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3913c-cc97-4947-b1d9-3406bed6ba3e None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:31.643396 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 234/458] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:31.648415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9375dd8-f0c8-435f-a43d-f0be269a51d6 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:31.648693 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9375dd8-f0c8-435f-a43d-f0be269a51d6 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:31.648962 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9375dd8-f0c8-435f-a43d-f0be269a51d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:31.661625 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c3913c-cc97-4947-b1d9-3406bed6ba3e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:31.679728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-c9375dd8-f0c8-435f-a43d-f0be269a51d6 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:31.680734 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c9375dd8-f0c8-435f-a43d-f0be269a51d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:31.697023 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:31.757717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 225/459] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:31.760070 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98029f8-05f7-42fa-8001-fa8688a49b9c None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:31.760353 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98029f8-05f7-42fa-8001-fa8688a49b9c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:31.760618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98029f8-05f7-42fa-8001-fa8688a49b9c None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:31.762120 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 226/460] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:31.768189 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde491bc-dc7a-4608-a324-42912ce65959 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:31.768465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde491bc-dc7a-4608-a324-42912ce65959 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:31.771544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde491bc-dc7a-4608-a324-42912ce65959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:31.796730 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c3913c-cc97-4947-b1d9-3406bed6ba3e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:31.809530 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-cde491bc-dc7a-4608-a324-42912ce65959 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:31.810733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-cde491bc-dc7a-4608-a324-42912ce65959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:31.859449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff87c7-aa9d-43a1-b9da-86188d92ff3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:31.860302 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff87c7-aa9d-43a1-b9da-86188d92ff3b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:31.860584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff87c7-aa9d-43a1-b9da-86188d92ff3b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:31.860852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff87c7-aa9d-43a1-b9da-86188d92ff3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:31.882527 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff87c7-aa9d-43a1-b9da-86188d92ff3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:31.884109 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff87c7-aa9d-43a1-b9da-86188d92ff3b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:31.887844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:31.922667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-89ff87c7-aa9d-43a1-b9da-86188d92ff3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:31.922667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:31.922667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:31.922667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:31.922667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:31.922667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:31.922667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:31.922667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:31.922667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:31.922667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:31.922667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:31.922667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:31.922667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:31.922667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:31.922667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:31.922667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:31.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:31.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:31.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:31.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:31.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:31.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:31.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:31.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:31.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:31.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:31.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:31.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:31.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:31.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:31.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:31.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:31.923658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:31.924903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:31.924903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:31.924903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:31.924903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 235/461] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:31.931857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff87c7-aa9d-43a1-b9da-86188d92ff3b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:31.953427 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 227/462] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:31.981083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a506054-5ef3-4bac-8497-26673d534a69 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:31.981419 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a506054-5ef3-4bac-8497-26673d534a69 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:31.981669 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a506054-5ef3-4bac-8497-26673d534a69 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:31.983261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 228/463] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:31.987871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b2e25-3e4b-45ac-82cc-d955be20eee0 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:31.988236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b2e25-3e4b-45ac-82cc-d955be20eee0 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:31.988667 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b2e25-3e4b-45ac-82cc-d955be20eee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:32.022669 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-af7b2e25-3e4b-45ac-82cc-d955be20eee0 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:32.026351 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-af7b2e25-3e4b-45ac-82cc-d955be20eee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:32.038455 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:32.073093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff87c7-aa9d-43a1-b9da-86188d92ff3b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:32.115953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb011f-c82c-44fb-bdc6-25e474239159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:32.117319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb011f-c82c-44fb-bdc6-25e474239159 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:32.117851 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb011f-c82c-44fb-bdc6-25e474239159 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:32.118372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb011f-c82c-44fb-bdc6-25e474239159 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:32.119253 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb011f-c82c-44fb-bdc6-25e474239159 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:32.119762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb011f-c82c-44fb-bdc6-25e474239159 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:32.121526 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb011f-c82c-44fb-bdc6-25e474239159 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:32.125599 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 229/464] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:05:32.126982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b2e25-3e4b-45ac-82cc-d955be20eee0 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:32.149252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 236/465] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:32.159878 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb011f-c82c-44fb-bdc6-25e474239159 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:32.236423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b2e25-3e4b-45ac-82cc-d955be20eee0 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:32.238251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:32.273208 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e642f-af2f-4161-be1a-01c3401ea2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:32.274115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e642f-af2f-4161-be1a-01c3401ea2ef None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:32.275062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e642f-af2f-4161-be1a-01c3401ea2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:32.275532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e642f-af2f-4161-be1a-01c3401ea2ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:32.286183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e642f-af2f-4161-be1a-01c3401ea2ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:32.287882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e642f-af2f-4161-be1a-01c3401ea2ef None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:32.303842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-892e642f-af2f-4161-be1a-01c3401ea2ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:32.303842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:32.303842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:32.303842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:32.303842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:32.303842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:32.303842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:32.303842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:32.303842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:32.303842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:32.303842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:32.303842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:32.303842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:32.303842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:32.303842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:32.303842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:32.304927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:32.304927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:32.304927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:32.304927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:32.304927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:32.304927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:32.304927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:32.304927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:32.304927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:32.304927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:32.304927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:32.304927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:32.304927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:32.304927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:32.304927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:32.304927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:32.304927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:32.305757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:32.305757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:32.305757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:32.307642 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 230/466] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:32.308661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e642f-af2f-4161-be1a-01c3401ea2ef None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:32.315041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb011f-c82c-44fb-bdc6-25e474239159 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:32.357894 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63bb31-a22b-4574-a4f3-7c600c60b420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:32.358857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63bb31-a22b-4574-a4f3-7c600c60b420 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:32.359575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63bb31-a22b-4574-a4f3-7c600c60b420 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:32.360532 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63bb31-a22b-4574-a4f3-7c600c60b420 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:32.361651 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63bb31-a22b-4574-a4f3-7c600c60b420 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:32.363977 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63bb31-a22b-4574-a4f3-7c600c60b420 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:32.380383 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:32.421445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e642f-af2f-4161-be1a-01c3401ea2ef None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:32.450259 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 237/467] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:05:32 2022] POST /identity/v3/users => generated 246 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:05:32.451829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63bb31-a22b-4574-a4f3-7c600c60b420 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:32.472382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29bedf-2b8e-4530-bd7a-4be9bac868f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:32.473026 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29bedf-2b8e-4530-bd7a-4be9bac868f5 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:32.473487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29bedf-2b8e-4530-bd7a-4be9bac868f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:32.473943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29bedf-2b8e-4530-bd7a-4be9bac868f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:32.490024 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29bedf-2b8e-4530-bd7a-4be9bac868f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:32.491769 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29bedf-2b8e-4530-bd7a-4be9bac868f5 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:32.508850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-1e29bedf-2b8e-4530-bd7a-4be9bac868f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:32.508850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:32.508850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:32.508850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:32.508850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:32.508850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:32.508850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:32.508850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:32.508850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:32.508850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:32.508850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:32.508850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:32.508850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:32.508850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:32.508850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:32.508850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:32.510343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:32.510343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:32.510343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:32.510343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:32.510343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:32.510343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:32.510343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:32.510343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:32.510343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:32.510343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:32.510343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:32.510343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:32.510343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:32.510343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:32.510343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:32.510343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:32.510343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:32.511653 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:32.511653 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:32.511653 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:32.511653 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 231/468] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:32.511862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29bedf-2b8e-4530-bd7a-4be9bac868f5 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:32.529604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:32.587882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63bb31-a22b-4574-a4f3-7c600c60b420 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:32.630883 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa05b4-fd52-42a2-806d-1723cae1bc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:32.631856 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa05b4-fd52-42a2-806d-1723cae1bc0b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:32.632352 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa05b4-fd52-42a2-806d-1723cae1bc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:32.632806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa05b4-fd52-42a2-806d-1723cae1bc0b None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:32.633646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa05b4-fd52-42a2-806d-1723cae1bc0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:32.634724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa05b4-fd52-42a2-806d-1723cae1bc0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:32.636470 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa05b4-fd52-42a2-806d-1723cae1bc0b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:32.653910 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 238/469] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:32.655132 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa05b4-fd52-42a2-806d-1723cae1bc0b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:32.672392 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29bedf-2b8e-4530-bd7a-4be9bac868f5 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:32.718522 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb87f9-5b1a-45d3-a446-cba361af6d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:32.719323 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb87f9-5b1a-45d3-a446-cba361af6d0a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:32.719677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb87f9-5b1a-45d3-a446-cba361af6d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:32.719956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb87f9-5b1a-45d3-a446-cba361af6d0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:32.728871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:32.732890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb87f9-5b1a-45d3-a446-cba361af6d0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:32.734623 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb87f9-5b1a-45d3-a446-cba361af6d0a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:32.751050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-cbfb87f9-5b1a-45d3-a446-cba361af6d0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:32.751050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:32.751050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:32.751050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:32.751050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:32.751050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:32.751050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:32.751050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:32.751050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:32.751050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:32.751050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:32.751050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:32.751050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:32.751050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:32.751050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:32.751050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:32.752118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:32.752118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:32.752118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:32.752118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:32.752118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:32.752118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:32.752118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:32.752118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:32.752118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:32.752118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:32.752118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:32.752118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:32.752118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:32.752118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:32.752118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:32.752118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:32.752118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:32.753219 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:32.753219 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:32.753219 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:32.753219 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 232/470] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:32.753426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb87f9-5b1a-45d3-a446-cba361af6d0a None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:32.794045 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa05b4-fd52-42a2-806d-1723cae1bc0b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:32.835000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b7e80-ae93-4c6c-8df6-dd076e826625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:32.835926 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b7e80-ae93-4c6c-8df6-dd076e826625 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:32.836211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b7e80-ae93-4c6c-8df6-dd076e826625 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:32.836756 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:32.837245 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b7e80-ae93-4c6c-8df6-dd076e826625 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:32.837919 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b7e80-ae93-4c6c-8df6-dd076e826625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:32.838337 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b7e80-ae93-4c6c-8df6-dd076e826625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:32.839790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b7e80-ae93-4c6c-8df6-dd076e826625 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:32.852957 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 239/471] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:32.855288 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b7e80-ae93-4c6c-8df6-dd076e826625 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:32.881918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb87f9-5b1a-45d3-a446-cba361af6d0a None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:32.917424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436c7f8-005b-4004-a02c-d28988b0f40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:32.918031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436c7f8-005b-4004-a02c-d28988b0f40e None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:32.918314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436c7f8-005b-4004-a02c-d28988b0f40e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:32.918589 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436c7f8-005b-4004-a02c-d28988b0f40e None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:32.919265 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436c7f8-005b-4004-a02c-d28988b0f40e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:32.920738 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436c7f8-005b-4004-a02c-d28988b0f40e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:32.961178 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b7e80-ae93-4c6c-8df6-dd076e826625 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:32.977972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:32.982420 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 233/472] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:05:32 2022] POST /identity/v3/users => generated 248 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:05:32.983699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e49ac-236b-4d13-aa46-afa9285f301c None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:32.983986 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e49ac-236b-4d13-aa46-afa9285f301c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:32.984262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e49ac-236b-4d13-aa46-afa9285f301c None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:32.985698 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 234/473] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:32.988065 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e49ac-236b-4d13-aa46-afa9285f301c None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:33.017900 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f68db9-79ea-436e-b205-dc0e34b48bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:33.017900 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f68db9-79ea-436e-b205-dc0e34b48bea None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:33.017900 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f68db9-79ea-436e-b205-dc0e34b48bea None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:33.017900 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f68db9-79ea-436e-b205-dc0e34b48bea None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:33.018795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f68db9-79ea-436e-b205-dc0e34b48bea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:33.019652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f68db9-79ea-436e-b205-dc0e34b48bea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:33.021922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f68db9-79ea-436e-b205-dc0e34b48bea None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:33.043444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 240/474] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:33.046324 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b785d34-9b3b-45fa-94fc-cbe62ecc0f24 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:33.046906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b785d34-9b3b-45fa-94fc-cbe62ecc0f24 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:33.047524 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b785d34-9b3b-45fa-94fc-cbe62ecc0f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:33.071336 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-6b785d34-9b3b-45fa-94fc-cbe62ecc0f24 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:33.073933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6b785d34-9b3b-45fa-94fc-cbe62ecc0f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:33.129285 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e49ac-236b-4d13-aa46-afa9285f301c None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:33.151304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:33.171343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 241/475] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:33.173462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b785d34-9b3b-45fa-94fc-cbe62ecc0f24 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:33.183036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efcde9-368c-4376-9f59-d9c021c4437c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:33.183743 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efcde9-368c-4376-9f59-d9c021c4437c None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:33.184019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efcde9-368c-4376-9f59-d9c021c4437c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:33.184342 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efcde9-368c-4376-9f59-d9c021c4437c None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:33.184994 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24efcde9-368c-4376-9f59-d9c021c4437c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:33.186475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24efcde9-368c-4376-9f59-d9c021c4437c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:33.293804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b785d34-9b3b-45fa-94fc-cbe62ecc0f24 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:33.303621 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:33.334258 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947468b-d836-4df3-92a4-21ae662334a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:33.335489 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947468b-d836-4df3-92a4-21ae662334a7 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:33.336027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947468b-d836-4df3-92a4-21ae662334a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:33.336476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947468b-d836-4df3-92a4-21ae662334a7 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:33.337361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947468b-d836-4df3-92a4-21ae662334a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:33.338443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 235/476] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:05:32 2022] POST /identity/v3/users => generated 248 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:05:33.339357 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947468b-d836-4df3-92a4-21ae662334a7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:33.437321 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 242/477] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:05:33 2022] POST /identity/v3/users => generated 249 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:05:33.456927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efcde9-368c-4376-9f59-d9c021c4437c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:33.598860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efcde9-368c-4376-9f59-d9c021c4437c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:33.639294 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcc857-a1b4-4147-b911-9090d63d0d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:33.639918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcc857-a1b4-4147-b911-9090d63d0d7d None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:33.640197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcc857-a1b4-4147-b911-9090d63d0d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:33.640470 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcc857-a1b4-4147-b911-9090d63d0d7d None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:33.641172 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedcc857-a1b4-4147-b911-9090d63d0d7d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:33.642632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedcc857-a1b4-4147-b911-9090d63d0d7d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:33.685604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.sql.core [None req-cedcc857-a1b4-4147-b911-9090d63d0d7d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 11:05:33.685604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 19 11:05:33.685604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [parameters: {'id': 'cde5f90ac2c14443b4ca4e0c6e1cc4e7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 19 11:05:33.685604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81738) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 11:05:33.705435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-cedcc857-a1b4-4147-b911-9090d63d0d7d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 19 11:05:33.705435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:33.705435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:05:33.705435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:05:33.705435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:05:33.705435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:05:33.705435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:05:33.705435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:05:33.705435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:05:33.705435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:05:33.705435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:05:33.705435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:05:33.705435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:05:33.705435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application result.read() Sep 19 11:05:33.705435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return ref.to_dict() Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:05:33.706391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:05:33.707334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 11:05:33.707334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application yield resource Sep 19 11:05:33.707334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:05:33.707334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:05:33.707334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 11:05:33.707334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 11:05:33.707334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 11:05:33.707334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 11:05:33.707334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 11:05:33.707334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise self.value Sep 19 11:05:33.707334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 11:05:33.707334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 11:05:33.707334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 11:05:33.707334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application session.commit() Sep 19 11:05:33.707334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 11:05:33.707334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 11:05:33.707334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 11:05:33.708203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 11:05:33.708203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 11:05:33.708203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self.session.flush() Sep 19 11:05:33.708203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 11:05:33.708203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 11:05:33.708203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 11:05:33.708203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 11:05:33.708203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 11:05:33.708203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application compat.raise_( Sep 19 11:05:33.708203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:05:33.708203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception Sep 19 11:05:33.708203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 11:05:33.708203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 11:05:33.708203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 11:05:33.708203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 11:05:33.708203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 11:05:33.708203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 11:05:33.709106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 11:05:33.709106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 11:05:33.709106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 19 11:05:33.709106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 19 11:05:33.709106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 11:05:33.709106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 11:05:33.709106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 11:05:33.709106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 11:05:33.709106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 11:05:33.709106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 11:05:33.709106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 11:05:33.709106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 11:05:33.709106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 11:05:33.709106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 11:05:33.709106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:05:33.709106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception Sep 19 11:05:33.709925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:05:33.709925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:05:33.709925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:05:33.709925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:05:33.709925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:05:33.709925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:05:33.709925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:05:33.709925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:05:33.709925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:05:33.709925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:05:33.709925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:05:33.709925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application result.read() Sep 19 11:05:33.709925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:05:33.709925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:05:33.709925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:05:33.709925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:05:33.710699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:05:33.710699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:05:33.710699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:05:33.710699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:05:33.710699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 11:05:33.710699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 19 11:05:33.710699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [parameters: {'id': 'cde5f90ac2c14443b4ca4e0c6e1cc4e7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 19 11:05:33.710699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 11:05:33.710699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:33.710699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:05:33.710699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:33.710699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:33.710699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:33.710699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:33.710699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:33.710699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:33.713305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:33.713305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:33.713305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:33.713305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:33.713305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:33.713305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:33.713305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 19 11:05:33.713305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 19 11:05:33.713305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:33.713305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:33.713305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 19 11:05:33.713305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 19 11:05:33.713305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 11:05:33.713305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 11:05:33.713305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 19 11:05:33.713305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:33.714337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 236/478] 10.222.0.78 () {60 vars in 1131 bytes} [Mon Sep 19 11:05:33 2022] POST /identity/v3/roles => generated 144 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:33.715319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947468b-d836-4df3-92a4-21ae662334a7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:33.832241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947468b-d836-4df3-92a4-21ae662334a7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:33.868485 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd8cb2-8ae2-40da-836b-5dd2546af90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:33.869348 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd8cb2-8ae2-40da-836b-5dd2546af90f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:33.869779 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd8cb2-8ae2-40da-836b-5dd2546af90f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:33.870212 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd8cb2-8ae2-40da-836b-5dd2546af90f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:33.881283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd8cb2-8ae2-40da-836b-5dd2546af90f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:33.887476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-30cd8cb2-8ae2-40da-836b-5dd2546af90f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:05:33.887476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:33.887476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:33.887476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:33.887476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:33.887476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:33.887476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:33.887476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:33.887476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:33.887476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:33.887476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:33.887476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:33.887476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:33.887476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:33.887476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:33.887476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:33.888552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:33.888552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:33.888552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:33.888552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:33.888552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:33.888552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:33.888552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:33.888552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:33.888552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:33.888552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:33.888552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:33.888552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:33.888552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:33.888552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:33.888552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:33.888552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:33.889540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:33.889540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:33.889540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:33.889540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:33.889540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:33.889540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:33.889540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:33.889540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:33.889540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:33.889540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:33.889540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:05:33.889540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:33.891834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 243/479] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:05:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:33.897805 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcc857-a1b4-4147-b911-9090d63d0d7d None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:33.937620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:34.010298 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcc857-a1b4-4147-b911-9090d63d0d7d None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:34.047540 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27a745-1c5b-418c-9b5b-4b716c3612c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:34.048226 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27a745-1c5b-418c-9b5b-4b716c3612c5 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:34.048512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27a745-1c5b-418c-9b5b-4b716c3612c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:34.048791 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27a745-1c5b-418c-9b5b-4b716c3612c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:34.049472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27a745-1c5b-418c-9b5b-4b716c3612c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:34.049838 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27a745-1c5b-418c-9b5b-4b716c3612c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:34.051319 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27a745-1c5b-418c-9b5b-4b716c3612c5 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:34.060964 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 237/480] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:34.969083 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda31b5-8f79-42b0-aca3-4e4644f07fc4 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:34.969602 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda31b5-8f79-42b0-aca3-4e4644f07fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:34.969602 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda31b5-8f79-42b0-aca3-4e4644f07fc4 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:34.971215 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 244/481] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:34.976908 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378bbd3b-d39a-4029-879a-8ff96e347416 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:34.978261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378bbd3b-d39a-4029-879a-8ff96e347416 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:34.978720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378bbd3b-d39a-4029-879a-8ff96e347416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:35.001994 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-378bbd3b-d39a-4029-879a-8ff96e347416 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:35.003771 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-378bbd3b-d39a-4029-879a-8ff96e347416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:35.122976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 238/482] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:35.544555 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda31b5-8f79-42b0-aca3-4e4644f07fc4 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:35.580988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:35.598620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d712fb-b78a-4d43-b429-36d66d509428 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:35.599004 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d712fb-b78a-4d43-b429-36d66d509428 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:35.599394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d712fb-b78a-4d43-b429-36d66d509428 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:35.601048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 239/483] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:35.606259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170de12-33a6-426c-8949-e761090f7f47 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:35.606607 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170de12-33a6-426c-8949-e761090f7f47 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:35.606950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170de12-33a6-426c-8949-e761090f7f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:35.626468 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-6170de12-33a6-426c-8949-e761090f7f47 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:35.628031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-6170de12-33a6-426c-8949-e761090f7f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:35.653244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:35.717682 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 240/484] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:35.727982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda31b5-8f79-42b0-aca3-4e4644f07fc4 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:35.778263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb01af-ebde-48a9-b106-02f4f889eea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:35.778911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb01af-ebde-48a9-b106-02f4f889eea6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:35.779198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb01af-ebde-48a9-b106-02f4f889eea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:35.779481 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb01af-ebde-48a9-b106-02f4f889eea6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:35.793584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb01af-ebde-48a9-b106-02f4f889eea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:35.798529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-ccfb01af-ebde-48a9-b106-02f4f889eea6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:35.798529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:35.798529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:35.798529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:35.798529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:35.798529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:35.798529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:35.798529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:35.798529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:35.798529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:35.798529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:35.798529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:35.798529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:35.798529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:35.798529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:35.798529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:35.799442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:35.799442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:35.799442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:35.799442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:35.799442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:35.799442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:35.799442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:35.799442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:35.799442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:35.799442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:35.799442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:35.799442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:35.799442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:35.799442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:35.799442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:35.799442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:35.800237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:35.800237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:35.800237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:35.800237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:35.800237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:35.800237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:35.800237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:35.800237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:35.800237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:35.800237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:35.800237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:35.800237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:35.801311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 245/485] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:35.806540 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170de12-33a6-426c-8949-e761090f7f47 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:35.940872 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170de12-33a6-426c-8949-e761090f7f47 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:35.984774 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec8bd4-b415-4d96-a832-5100a9bdf4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:35.985397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec8bd4-b415-4d96-a832-5100a9bdf4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:35.985650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec8bd4-b415-4d96-a832-5100a9bdf4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:35.985916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec8bd4-b415-4d96-a832-5100a9bdf4c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:35.986578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec8bd4-b415-4d96-a832-5100a9bdf4c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:35.986875 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec8bd4-b415-4d96-a832-5100a9bdf4c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:35.988262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec8bd4-b415-4d96-a832-5100a9bdf4c8 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:36.000071 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb01af-ebde-48a9-b106-02f4f889eea6 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:36.003370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 241/486] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:05:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:36.010597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec8bd4-b415-4d96-a832-5100a9bdf4c8 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:36.043578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:36.127013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:36.161827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb01af-ebde-48a9-b106-02f4f889eea6 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:36.199747 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec8bd4-b415-4d96-a832-5100a9bdf4c8 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:36.228169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d88ea8-e3c7-4d9a-90d0-aba2aec5ba0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:36.228807 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d88ea8-e3c7-4d9a-90d0-aba2aec5ba0c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:36.229082 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d88ea8-e3c7-4d9a-90d0-aba2aec5ba0c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:36.229350 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d88ea8-e3c7-4d9a-90d0-aba2aec5ba0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:36.249330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d88ea8-e3c7-4d9a-90d0-aba2aec5ba0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:36.255364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-52d88ea8-e3c7-4d9a-90d0-aba2aec5ba0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:36.255364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:36.255364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:36.255364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:36.255364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:36.255364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:36.255364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:36.255364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:36.255364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:36.255364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:36.255364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:36.255364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:36.255364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:36.255364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:36.255364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:36.255364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:36.256236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:36.256236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:36.256236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:36.256236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:36.256236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:36.256236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:36.256236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:36.256236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:36.256236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:36.256236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:36.256236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:36.256236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:36.256236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:36.256236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:36.256236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:36.256236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:36.257029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:36.257029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:36.257029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:36.257029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:36.257029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:36.257029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:36.257029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:36.257029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:36.257029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:36.257029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:36.257029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:36.257029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:36.261118 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 246/487] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:36.267630 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d88ea8-e3c7-4d9a-90d0-aba2aec5ba0c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:36.269935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c56e7-2963-4683-b0f0-355945fbaa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:36.270732 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c56e7-2963-4683-b0f0-355945fbaa11 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:36.271180 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c56e7-2963-4683-b0f0-355945fbaa11 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:36.271590 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c56e7-2963-4683-b0f0-355945fbaa11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:36.293895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c56e7-2963-4683-b0f0-355945fbaa11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:36.295646 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c56e7-2963-4683-b0f0-355945fbaa11 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:36.314661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-846c56e7-2963-4683-b0f0-355945fbaa11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:36.314661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:36.314661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:36.314661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:36.314661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:36.314661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:36.314661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:36.314661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:36.314661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:36.314661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:36.314661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:36.314661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:36.314661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:36.314661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:36.314661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:36.314661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:36.315842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:36.315842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:36.315842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:36.315842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:36.315842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:36.315842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:36.315842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:36.315842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:36.315842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:36.315842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:36.315842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:36.315842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:36.315842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:36.315842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:36.315842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:36.315842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:36.315842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:36.316895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:36.316895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:36.316895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:36.318597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 242/488] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:36.324145 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c56e7-2963-4683-b0f0-355945fbaa11 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:36.470518 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:36.470689 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d88ea8-e3c7-4d9a-90d0-aba2aec5ba0c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:36.510948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9f408-172d-4fb9-bbb3-fe2b1a7fdcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:36.511153 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c56e7-2963-4683-b0f0-355945fbaa11 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:36.511608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9f408-172d-4fb9-bbb3-fe2b1a7fdcc8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:36.511894 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9f408-172d-4fb9-bbb3-fe2b1a7fdcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:36.512173 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9f408-172d-4fb9-bbb3-fe2b1a7fdcc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:36.512887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9f408-172d-4fb9-bbb3-fe2b1a7fdcc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:36.513193 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9f408-172d-4fb9-bbb3-fe2b1a7fdcc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:36.514580 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9f408-172d-4fb9-bbb3-fe2b1a7fdcc8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:36.523665 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 247/489] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:05:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:36.525489 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b859ee7-ac0a-4f42-9269-c1623bdf1d5b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:36.525781 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b859ee7-ac0a-4f42-9269-c1623bdf1d5b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:36.526056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b859ee7-ac0a-4f42-9269-c1623bdf1d5b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:36.527613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 248/490] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:36.530499 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b859ee7-ac0a-4f42-9269-c1623bdf1d5b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:36.534371 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:36.555288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab6772a-3c05-4b46-89d0-e6b767ce927b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:36.556431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6772a-3c05-4b46-89d0-e6b767ce927b None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:36.556710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6772a-3c05-4b46-89d0-e6b767ce927b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:36.557446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6772a-3c05-4b46-89d0-e6b767ce927b None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:36.558105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab6772a-3c05-4b46-89d0-e6b767ce927b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:36.558854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab6772a-3c05-4b46-89d0-e6b767ce927b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:36.560475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab6772a-3c05-4b46-89d0-e6b767ce927b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:36.585492 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 243/491] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:36.587507 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab03195-e566-4195-aa68-7184e54b057d None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:36.587949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab03195-e566-4195-aa68-7184e54b057d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:36.588349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab03195-e566-4195-aa68-7184e54b057d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:36.613001 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-0ab03195-e566-4195-aa68-7184e54b057d None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:36.614585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-0ab03195-e566-4195-aa68-7184e54b057d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:36.659138 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b859ee7-ac0a-4f42-9269-c1623bdf1d5b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:36.699662 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15e840-c9b2-4e3c-8b4a-73782ca3aceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:36.700522 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15e840-c9b2-4e3c-8b4a-73782ca3aceb None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:36.700908 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15e840-c9b2-4e3c-8b4a-73782ca3aceb None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:36.701287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15e840-c9b2-4e3c-8b4a-73782ca3aceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:36.716342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15e840-c9b2-4e3c-8b4a-73782ca3aceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:36.718136 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15e840-c9b2-4e3c-8b4a-73782ca3aceb None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:36.721318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 244/492] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:36.722009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab03195-e566-4195-aa68-7184e54b057d None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:36.734450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-aa15e840-c9b2-4e3c-8b4a-73782ca3aceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:36.734450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:36.734450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:36.734450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:36.734450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:36.734450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:36.734450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:36.734450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:36.734450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:36.734450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:36.734450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:36.734450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:36.734450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:36.734450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:36.734450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:36.734450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:36.736767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:36.736767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:36.736767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:36.736767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:36.736767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:36.736767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:36.736767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:36.736767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:36.736767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:36.736767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:36.736767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:36.736767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:36.736767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:36.736767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:36.736767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:36.736767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:36.736767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:36.739090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:36.739090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:36.739090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:36.739090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 249/493] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:36.739090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d903c-3a48-4f1d-9396-944fffc10d65 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:36.739090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d903c-3a48-4f1d-9396-944fffc10d65 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:36.739090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d903c-3a48-4f1d-9396-944fffc10d65 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:36.744669 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 250/494] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:36.748362 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d903c-3a48-4f1d-9396-944fffc10d65 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:36.822976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:36.827743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:36.874576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab03195-e566-4195-aa68-7184e54b057d None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:36.881788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d903c-3a48-4f1d-9396-944fffc10d65 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:36.910921 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84488b3-5b28-45eb-85bf-17d81afca5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:36.911585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84488b3-5b28-45eb-85bf-17d81afca5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:36.911858 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84488b3-5b28-45eb-85bf-17d81afca5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:36.912127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84488b3-5b28-45eb-85bf-17d81afca5b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:36.918459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e10392-10eb-43cc-9364-2ef046c6c5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:36.919073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e10392-10eb-43cc-9364-2ef046c6c5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:36.919383 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e10392-10eb-43cc-9364-2ef046c6c5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:36.919661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e10392-10eb-43cc-9364-2ef046c6c5c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:36.919989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84488b3-5b28-45eb-85bf-17d81afca5b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:36.920309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e10392-10eb-43cc-9364-2ef046c6c5c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:36.920616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e10392-10eb-43cc-9364-2ef046c6c5c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:36.921433 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84488b3-5b28-45eb-85bf-17d81afca5b1 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:36.921987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e10392-10eb-43cc-9364-2ef046c6c5c4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:36.938760 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 251/495] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:36.938937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-e84488b3-5b28-45eb-85bf-17d81afca5b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:05:36.938937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:36.938937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:36.938937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:36.938937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:36.938937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:36.938937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:36.938937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:36.938937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:36.938937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:36.938937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:36.938937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:36.938937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:05:36.938937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:05:36.938937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:05:36.938937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:36.939880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:36.940792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:36.940792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:36.940792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:05:36.940792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:05:36.940792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:05:36.940792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:05:36.940792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:05:36.940792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:05:36.940792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:05:36.940792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:36.941304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc33929-7b2d-434b-8714-bd1867dc42be None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:36.941304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc33929-7b2d-434b-8714-bd1867dc42be None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:36.941304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc33929-7b2d-434b-8714-bd1867dc42be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:36.941480 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 245/496] 10.222.0.78 () {56 vars in 1093 bytes} [Mon Sep 19 11:05:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:36.941480 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856fdb1-fd6e-4c5f-96d0-15949d24ca14 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:36.941480 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856fdb1-fd6e-4c5f-96d0-15949d24ca14 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:36.941480 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856fdb1-fd6e-4c5f-96d0-15949d24ca14 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:36.942892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 246/497] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:36.943539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856fdb1-fd6e-4c5f-96d0-15949d24ca14 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:36.959264 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-7bc33929-7b2d-434b-8714-bd1867dc42be None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:36.960571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7bc33929-7b2d-434b-8714-bd1867dc42be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:37.032240 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 252/498] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:37.036098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc33929-7b2d-434b-8714-bd1867dc42be None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:37.069121 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856fdb1-fd6e-4c5f-96d0-15949d24ca14 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:37.083163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:37.092753 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:37.107148 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ecf9e-24ab-4c70-98b5-f85be81a87bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:37.107814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ecf9e-24ab-4c70-98b5-f85be81a87bf None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:37.110558 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ecf9e-24ab-4c70-98b5-f85be81a87bf None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:37.110558 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ecf9e-24ab-4c70-98b5-f85be81a87bf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:37.117705 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ecf9e-24ab-4c70-98b5-f85be81a87bf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:37.119874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ecf9e-24ab-4c70-98b5-f85be81a87bf None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:37.132251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-fe0ecf9e-24ab-4c70-98b5-f85be81a87bf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:05:37.132251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:37.132251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:37.132251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:37.132251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:37.132251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:37.132251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:37.132251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:37.132251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:37.132251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:37.132251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:37.132251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:37.132251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:05:37.132251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:05:37.132251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:05:37.132251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:37.138982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:37.140127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:37.140127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:37.140127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:05:37.140127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:05:37.140127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:05:37.140127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:05:37.140127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:05:37.140127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:05:37.140127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:05:37.140127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:37.140127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 247/499] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:37.140127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1965b8-5f8b-4a67-b809-f5d8ddcdffb6 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:37.140127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1965b8-5f8b-4a67-b809-f5d8ddcdffb6 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:37.141015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1965b8-5f8b-4a67-b809-f5d8ddcdffb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:37.157196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-3d1965b8-5f8b-4a67-b809-f5d8ddcdffb6 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:37.158641 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-3d1965b8-5f8b-4a67-b809-f5d8ddcdffb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:37.167911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc33929-7b2d-434b-8714-bd1867dc42be None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:37.204782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1e50c-9f00-4f03-924d-abc2879979ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:37.205410 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1e50c-9f00-4f03-924d-abc2879979ff None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:37.205680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1e50c-9f00-4f03-924d-abc2879979ff None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:37.205963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1e50c-9f00-4f03-924d-abc2879979ff None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:37.206692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1e50c-9f00-4f03-924d-abc2879979ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:37.207007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1e50c-9f00-4f03-924d-abc2879979ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:37.208469 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1e50c-9f00-4f03-924d-abc2879979ff None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:37.239196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 248/500] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:37.239196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1965b8-5f8b-4a67-b809-f5d8ddcdffb6 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:37.253947 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 253/501] 10.222.0.78 () {56 vars in 1133 bytes} [Mon Sep 19 11:05:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:37.254935 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1e50c-9f00-4f03-924d-abc2879979ff None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:37.353607 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1965b8-5f8b-4a67-b809-f5d8ddcdffb6 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:37.364426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:37.372858 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1e50c-9f00-4f03-924d-abc2879979ff None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:37.376147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:37.387613 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca45fa8-b046-46f5-b20b-be273c5a438d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:37.388249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca45fa8-b046-46f5-b20b-be273c5a438d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:37.388524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca45fa8-b046-46f5-b20b-be273c5a438d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:37.388792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca45fa8-b046-46f5-b20b-be273c5a438d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:37.402765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca45fa8-b046-46f5-b20b-be273c5a438d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:37.407680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-8ca45fa8-b046-46f5-b20b-be273c5a438d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:37.407680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:37.407680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:37.407680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:37.407680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:37.407680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:37.407680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:37.407680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:37.407680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:37.407680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:37.407680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:37.407680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:37.407680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:37.407680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:37.407680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:37.407680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:37.408550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:37.408550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:37.408550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:37.408550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:37.408550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:37.408550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:37.408550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:37.408550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:37.408550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:37.408550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:37.408550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:37.408550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:37.408550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:37.408550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:37.408550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:37.408550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:37.409417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:37.409417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:37.409417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:37.409417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:37.409417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:37.409417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:37.409417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:37.409417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:37.409417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:37.409417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:37.409417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:37.409417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:37.409417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 249/502] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:37.410716 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca45fa8-b046-46f5-b20b-be273c5a438d None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:37.415688 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56319059-4f57-4aa7-8a7c-221087a60f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:37.416313 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56319059-4f57-4aa7-8a7c-221087a60f7a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:37.416591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56319059-4f57-4aa7-8a7c-221087a60f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:37.416861 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56319059-4f57-4aa7-8a7c-221087a60f7a None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:37.417604 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56319059-4f57-4aa7-8a7c-221087a60f7a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:37.417904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56319059-4f57-4aa7-8a7c-221087a60f7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:37.419500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56319059-4f57-4aa7-8a7c-221087a60f7a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:37.444029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 254/503] 10.222.0.78 () {56 vars in 1137 bytes} [Mon Sep 19 11:05:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:37.445034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56319059-4f57-4aa7-8a7c-221087a60f7a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:37.525032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca45fa8-b046-46f5-b20b-be273c5a438d None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:37.556315 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56319059-4f57-4aa7-8a7c-221087a60f7a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:37.564471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c55246-a6d9-4bd8-a2b1-2652a3f8ade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:37.565088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c55246-a6d9-4bd8-a2b1-2652a3f8ade6 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:37.565371 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c55246-a6d9-4bd8-a2b1-2652a3f8ade6 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:37.565649 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c55246-a6d9-4bd8-a2b1-2652a3f8ade6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:37.592249 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16872fd9-652d-4ce7-987d-ed4170be7a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:37.592885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16872fd9-652d-4ce7-987d-ed4170be7a56 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:37.593171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16872fd9-652d-4ce7-987d-ed4170be7a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:37.593453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16872fd9-652d-4ce7-987d-ed4170be7a56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:37.604085 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16872fd9-652d-4ce7-987d-ed4170be7a56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:37.606187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:37.608871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-16872fd9-652d-4ce7-987d-ed4170be7a56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:37.608871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:37.608871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:37.608871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:37.608871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:37.608871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:37.608871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:37.608871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:37.608871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:37.608871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:37.608871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:37.608871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:37.608871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:37.608871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:37.608871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:37.608871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:37.610759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:37.610759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:37.610759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:37.610759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:37.610759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:37.610759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:37.610759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:37.610759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:37.610759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:37.610759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:37.610759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:37.610759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:37.610759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:37.610759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:37.610759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:37.610759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:37.611805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:37.611805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:37.611805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:37.611805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:37.611805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:37.611805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:37.611805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:37.611805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:37.611805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:37.611805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:37.611805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:37.611805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:37.611805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 255/504] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:37.611805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16872fd9-652d-4ce7-987d-ed4170be7a56 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:37.638415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c55246-a6d9-4bd8-a2b1-2652a3f8ade6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:37.639443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:37.640223 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c55246-a6d9-4bd8-a2b1-2652a3f8ade6 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:37.713355 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16872fd9-652d-4ce7-987d-ed4170be7a56 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:37.738070 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 250/505] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:37.739029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c55246-a6d9-4bd8-a2b1-2652a3f8ade6 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:37.749122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433e19a-3b7b-49d6-b18c-30f5c7f8a760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:37.749730 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433e19a-3b7b-49d6-b18c-30f5c7f8a760 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:37.750020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433e19a-3b7b-49d6-b18c-30f5c7f8a760 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:37.750295 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433e19a-3b7b-49d6-b18c-30f5c7f8a760 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:37.750962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433e19a-3b7b-49d6-b18c-30f5c7f8a760 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:37.751304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433e19a-3b7b-49d6-b18c-30f5c7f8a760 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:37.752950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433e19a-3b7b-49d6-b18c-30f5c7f8a760 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:37.761662 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 256/506] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:05:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:37.762877 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433e19a-3b7b-49d6-b18c-30f5c7f8a760 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:37.807549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:37.837141 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c55246-a6d9-4bd8-a2b1-2652a3f8ade6 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:37.846893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:37.864227 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433e19a-3b7b-49d6-b18c-30f5c7f8a760 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:37.870969 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386af98d-8419-47de-b42f-7dbfe2e186ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:37.871609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386af98d-8419-47de-b42f-7dbfe2e186ef None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:37.871891 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386af98d-8419-47de-b42f-7dbfe2e186ef None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:37.872169 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386af98d-8419-47de-b42f-7dbfe2e186ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:37.899828 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d254d5-9d4b-4677-bc8b-04f5aea7e014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:37.900491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d254d5-9d4b-4677-bc8b-04f5aea7e014 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:37.900768 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d254d5-9d4b-4677-bc8b-04f5aea7e014 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:37.901043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d254d5-9d4b-4677-bc8b-04f5aea7e014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:37.911858 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d254d5-9d4b-4677-bc8b-04f5aea7e014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:37.913319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d254d5-9d4b-4677-bc8b-04f5aea7e014 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:37.930419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-d6d254d5-9d4b-4677-bc8b-04f5aea7e014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:37.930419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:37.930419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:37.930419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:37.930419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:37.930419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:37.930419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:37.930419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:37.930419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:37.930419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:37.930419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:37.930419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:37.930419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:37.930419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:37.930419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:37.930419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:37.932507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:37.932507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:37.932507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:37.932507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:37.932507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:37.932507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:37.932507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:37.932507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:37.932507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:37.932507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:37.932507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:37.932507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:37.932507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:37.932507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:37.932507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:37.932507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:37.932507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:37.933838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:37.933838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:37.933838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:37.933838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 257/507] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:37.933838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d254d5-9d4b-4677-bc8b-04f5aea7e014 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:37.944489 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386af98d-8419-47de-b42f-7dbfe2e186ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:37.945969 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386af98d-8419-47de-b42f-7dbfe2e186ef None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:38.016412 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:38.035864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d254d5-9d4b-4677-bc8b-04f5aea7e014 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:38.049887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 251/508] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:38.050737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386af98d-8419-47de-b42f-7dbfe2e186ef None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:38.073844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1bb26-f9ec-4080-b2e3-c0b723944246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:38.074448 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1bb26-f9ec-4080-b2e3-c0b723944246 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:38.074726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1bb26-f9ec-4080-b2e3-c0b723944246 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:38.074996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1bb26-f9ec-4080-b2e3-c0b723944246 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:38.075681 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1bb26-f9ec-4080-b2e3-c0b723944246 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:38.075980 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1bb26-f9ec-4080-b2e3-c0b723944246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:38.077343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1bb26-f9ec-4080-b2e3-c0b723944246 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:38.079463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:38.086233 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 258/509] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:05:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:38.087533 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1bb26-f9ec-4080-b2e3-c0b723944246 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:38.151940 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386af98d-8419-47de-b42f-7dbfe2e186ef None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:38.188132 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1bb26-f9ec-4080-b2e3-c0b723944246 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:38.224484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f5772-4246-44fe-a5db-72f6055aaf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:38.224723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ab847-7a8f-4d52-9ef4-608fa44006a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:38.225115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f5772-4246-44fe-a5db-72f6055aaf86 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:38.225358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ab847-7a8f-4d52-9ef4-608fa44006a3 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:38.225453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f5772-4246-44fe-a5db-72f6055aaf86 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:38.225638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ab847-7a8f-4d52-9ef4-608fa44006a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:38.225726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f5772-4246-44fe-a5db-72f6055aaf86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:38.225915 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ab847-7a8f-4d52-9ef4-608fa44006a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:38.236886 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ab847-7a8f-4d52-9ef4-608fa44006a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:38.238354 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ab847-7a8f-4d52-9ef4-608fa44006a3 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:38.243672 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f5772-4246-44fe-a5db-72f6055aaf86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:38.245107 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f5772-4246-44fe-a5db-72f6055aaf86 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:38.253308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-0c4ab847-7a8f-4d52-9ef4-608fa44006a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:38.253308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:38.253308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:38.253308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:38.253308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:38.253308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:38.253308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:38.253308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:38.253308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:38.253308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:38.253308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:38.253308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:38.253308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:38.253308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:38.253308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:38.253308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:38.254384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:38.254384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:38.254384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:38.254384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:38.254384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:38.254384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:38.254384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:38.254384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:38.254384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:38.254384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:38.254384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:38.254384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:38.254384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:38.254384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:38.254384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:38.254384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:38.254384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:38.255491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:38.255491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:38.255491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:38.255491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 252/510] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:38.255491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ab847-7a8f-4d52-9ef4-608fa44006a3 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:38.263120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-988f5772-4246-44fe-a5db-72f6055aaf86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:38.263120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:38.263120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:38.263120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:38.263120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:38.263120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:38.263120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:38.263120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:38.263120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:38.263120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:38.263120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:38.263120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:38.263120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:38.263120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:38.263120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:38.263120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:38.264027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:38.264027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:38.264027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:38.264027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:38.264027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:38.264027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:38.264027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:38.264027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:38.264027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:38.264027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:38.264027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:38.264027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:38.264027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:38.264027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:38.264027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:38.264027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:38.264027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:38.265093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:38.265093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:38.265093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:38.265093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 259/511] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:38.265093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f5772-4246-44fe-a5db-72f6055aaf86 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:38.284552 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:38.314008 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:38.362066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ab847-7a8f-4d52-9ef4-608fa44006a3 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:38.367192 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f5772-4246-44fe-a5db-72f6055aaf86 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:38.398695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7709e4c-b624-4981-8f2e-687013698cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:38.400170 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7709e4c-b624-4981-8f2e-687013698cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:38.400459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7709e4c-b624-4981-8f2e-687013698cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:38.400732 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7709e4c-b624-4981-8f2e-687013698cb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:38.401381 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7709e4c-b624-4981-8f2e-687013698cb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:38.401684 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7709e4c-b624-4981-8f2e-687013698cb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:38.403118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7709e4c-b624-4981-8f2e-687013698cb6 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:38.404785 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdad41a-f457-454a-8b8a-cf63a6b35ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:38.405433 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdad41a-f457-454a-8b8a-cf63a6b35ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:38.405741 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdad41a-f457-454a-8b8a-cf63a6b35ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:38.405991 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdad41a-f457-454a-8b8a-cf63a6b35ad4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:38.415239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 253/512] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:38.416078 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7709e4c-b624-4981-8f2e-687013698cb6 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:38.422438 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdad41a-f457-454a-8b8a-cf63a6b35ad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:38.423966 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdad41a-f457-454a-8b8a-cf63a6b35ad4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:38.438460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-ccdad41a-f457-454a-8b8a-cf63a6b35ad4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:38.438460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:38.438460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:38.438460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:38.438460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:38.438460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:38.438460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:38.438460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:38.438460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:38.438460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:38.438460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:38.438460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:38.438460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:38.438460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:38.438460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:38.438460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:38.439999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:38.439999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:38.439999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:38.439999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:38.439999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:38.439999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:38.439999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:38.439999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:38.439999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:38.439999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:38.439999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:38.439999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:38.439999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:38.439999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:38.439999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:38.439999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:38.439999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:38.440981 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:38.440981 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:38.440981 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:38.440981 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 260/513] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:38.440981 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdad41a-f457-454a-8b8a-cf63a6b35ad4 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:38.505142 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:38.522810 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7709e4c-b624-4981-8f2e-687013698cb6 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:38.542838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdad41a-f457-454a-8b8a-cf63a6b35ad4 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:38.544430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:38.557589 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6b79a-8dfa-4b2a-bad4-0f8a6311825b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:38.558223 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6b79a-8dfa-4b2a-bad4-0f8a6311825b None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:38.558507 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6b79a-8dfa-4b2a-bad4-0f8a6311825b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:38.558796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6b79a-8dfa-4b2a-bad4-0f8a6311825b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:38.568603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf6b79a-8dfa-4b2a-bad4-0f8a6311825b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:38.570053 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf6b79a-8dfa-4b2a-bad4-0f8a6311825b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:38.577948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910f6ff-1ae3-4f31-9df8-81e65fd12d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:38.578609 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910f6ff-1ae3-4f31-9df8-81e65fd12d06 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:38.578988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910f6ff-1ae3-4f31-9df8-81e65fd12d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:38.579304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910f6ff-1ae3-4f31-9df8-81e65fd12d06 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:38.579952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910f6ff-1ae3-4f31-9df8-81e65fd12d06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:38.580263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910f6ff-1ae3-4f31-9df8-81e65fd12d06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:38.581722 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910f6ff-1ae3-4f31-9df8-81e65fd12d06 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:38.585273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-aaf6b79a-8dfa-4b2a-bad4-0f8a6311825b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:38.585273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:38.585273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:38.585273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:38.585273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:38.585273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:38.585273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:38.585273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:38.585273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:38.585273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:38.585273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:38.585273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:38.585273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:38.585273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:38.585273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:38.585273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:38.586206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:38.586206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:38.586206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:38.586206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:38.586206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:38.586206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:38.586206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:38.586206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:38.586206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:38.586206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:38.586206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:38.586206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:38.586206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:38.586206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:38.586206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:38.586206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:38.586206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:38.587801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:38.587801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:38.587801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:38.587801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 254/514] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:38.587801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6b79a-8dfa-4b2a-bad4-0f8a6311825b None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:38.595477 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 261/515] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:38.596178 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910f6ff-1ae3-4f31-9df8-81e65fd12d06 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:38.693605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6b79a-8dfa-4b2a-bad4-0f8a6311825b None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:38.708364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910f6ff-1ae3-4f31-9df8-81e65fd12d06 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:38.729470 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52c8ab-433d-4bba-9864-647816c709a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:38.730092 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52c8ab-433d-4bba-9864-647816c709a1 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:38.730467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52c8ab-433d-4bba-9864-647816c709a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:38.730768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52c8ab-433d-4bba-9864-647816c709a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:38.731442 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52c8ab-433d-4bba-9864-647816c709a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:38.731742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52c8ab-433d-4bba-9864-647816c709a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:38.733157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52c8ab-433d-4bba-9864-647816c709a1 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:38.734819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:38.744459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050aa90-243f-4be5-a394-451b82878f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:38.744875 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050aa90-243f-4be5-a394-451b82878f60 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:38.745154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050aa90-243f-4be5-a394-451b82878f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:38.745421 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050aa90-243f-4be5-a394-451b82878f60 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:38.750211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 255/516] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:38.750525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050aa90-243f-4be5-a394-451b82878f60 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:38.750624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52c8ab-433d-4bba-9864-647816c709a1 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:38.750709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050aa90-243f-4be5-a394-451b82878f60 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:38.841231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 262/517] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:05:38 2022] POST /identity/v3/users => generated 247 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:05:38.856312 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050aa90-243f-4be5-a394-451b82878f60 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:38.862703 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52c8ab-433d-4bba-9864-647816c709a1 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:38.884036 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:38.904597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e1e54-8c9c-433f-a2eb-3675f24da852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:38.905262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e1e54-8c9c-433f-a2eb-3675f24da852 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:38.905548 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e1e54-8c9c-433f-a2eb-3675f24da852 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:38.905815 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e1e54-8c9c-433f-a2eb-3675f24da852 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:38.906485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e1e54-8c9c-433f-a2eb-3675f24da852 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:38.906860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e1e54-8c9c-433f-a2eb-3675f24da852 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:38.908342 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e1e54-8c9c-433f-a2eb-3675f24da852 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:38.922176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 256/518] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:38.922971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e1e54-8c9c-433f-a2eb-3675f24da852 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:38.984988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050aa90-243f-4be5-a394-451b82878f60 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:38.988966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:39.026571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66fa44-2318-414e-8f0b-525523d85245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:39.027429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66fa44-2318-414e-8f0b-525523d85245 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:39.028347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66fa44-2318-414e-8f0b-525523d85245 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:39.028347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66fa44-2318-414e-8f0b-525523d85245 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:39.028685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66fa44-2318-414e-8f0b-525523d85245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:39.028995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66fa44-2318-414e-8f0b-525523d85245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:39.030380 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66fa44-2318-414e-8f0b-525523d85245 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:39.042932 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e1e54-8c9c-433f-a2eb-3675f24da852 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:39.043732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 263/519] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:39.044459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66fa44-2318-414e-8f0b-525523d85245 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:39.088292 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ff1fe-f804-44fa-b520-bd778826d466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:39.088935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ff1fe-f804-44fa-b520-bd778826d466 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:39.089214 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ff1fe-f804-44fa-b520-bd778826d466 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:39.089486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ff1fe-f804-44fa-b520-bd778826d466 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:39.096855 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ff1fe-f804-44fa-b520-bd778826d466 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:39.098310 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ff1fe-f804-44fa-b520-bd778826d466 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:39.116478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-ea9ff1fe-f804-44fa-b520-bd778826d466 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:39.116478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:39.116478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:39.116478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:39.116478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:39.116478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:39.116478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:39.116478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:39.116478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:39.116478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:39.116478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:39.116478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:39.116478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:05:39.116478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:05:39.116478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:05:39.117300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:05:39.117300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:39.117300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:39.117300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:39.117300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:39.117300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:39.117300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:39.117300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:39.117300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:39.117300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:39.117300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:39.117300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:39.117300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:39.117300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:39.117300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:39.117300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:39.117300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:39.119282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:05:39.119282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:05:39.119282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:05:39.119282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:05:39.119282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:05:39.119282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:05:39.119282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:39.119282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:39.119282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 257/520] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:39.119778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ff1fe-f804-44fa-b520-bd778826d466 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:39.140546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:39.163858 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66fa44-2318-414e-8f0b-525523d85245 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:39.205907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0357c5b-7055-41ba-b024-f3e575d8da52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:39.206535 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0357c5b-7055-41ba-b024-f3e575d8da52 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:39.206817 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0357c5b-7055-41ba-b024-f3e575d8da52 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:39.207089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0357c5b-7055-41ba-b024-f3e575d8da52 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:39.216109 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0357c5b-7055-41ba-b024-f3e575d8da52 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:39.217701 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0357c5b-7055-41ba-b024-f3e575d8da52 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:39.231097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-b0357c5b-7055-41ba-b024-f3e575d8da52 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:05:39.231097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:39.231097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:39.231097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:39.231097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:39.231097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:39.231097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:39.231097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:39.231097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:39.231097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:39.231097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:39.231097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:39.231097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:05:39.231097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:05:39.231097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:05:39.231097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:39.232168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:39.233995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:39.233995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:39.233995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:05:39.233995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:05:39.233995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:05:39.233995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:05:39.233995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:05:39.233995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:05:39.233995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:05:39.233995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:39.233995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 264/521] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:39.233995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0357c5b-7055-41ba-b024-f3e575d8da52 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:39.239502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ff1fe-f804-44fa-b520-bd778826d466 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:39.255118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:39.274800 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1ce8f-9534-4162-82fa-8fde604422c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:39.275463 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1ce8f-9534-4162-82fa-8fde604422c5 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:39.275744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1ce8f-9534-4162-82fa-8fde604422c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:39.276017 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1ce8f-9534-4162-82fa-8fde604422c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:39.282804 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1ce8f-9534-4162-82fa-8fde604422c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:39.284494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1ce8f-9534-4162-82fa-8fde604422c5 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:39.297469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-71f1ce8f-9534-4162-82fa-8fde604422c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:39.297469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:39.297469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:39.297469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:39.297469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:39.297469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:39.297469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:39.297469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:39.297469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:39.297469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:39.297469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:39.297469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:39.297469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:05:39.297469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:05:39.297469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:05:39.298384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:05:39.298384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:39.298384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:39.298384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:39.298384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:39.298384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:39.298384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:39.298384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:39.298384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:39.298384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:39.298384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:39.298384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:39.298384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:39.298384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:39.298384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:39.298384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:39.298384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:39.299894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:05:39.299894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:05:39.299894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:05:39.299894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:05:39.299894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:05:39.299894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:05:39.299894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:39.299894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:39.299894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 258/522] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:39.299894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1ce8f-9534-4162-82fa-8fde604422c5 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:39.345927 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0357c5b-7055-41ba-b024-f3e575d8da52 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:39.381703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb201c50-c764-4609-8a75-733e93654cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:39.382339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb201c50-c764-4609-8a75-733e93654cee None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:39.382620 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb201c50-c764-4609-8a75-733e93654cee None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:39.382896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb201c50-c764-4609-8a75-733e93654cee None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:39.390638 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb201c50-c764-4609-8a75-733e93654cee None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:39.392120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb201c50-c764-4609-8a75-733e93654cee None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:39.394429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:39.404702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-bb201c50-c764-4609-8a75-733e93654cee None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:05:39.404702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:39.404702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:39.404702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:39.404702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:39.404702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:39.404702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:39.404702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:39.404702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:39.404702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:39.404702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:39.404702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:39.404702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:05:39.404702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:05:39.404702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:05:39.404702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:39.406401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:39.407501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:39.407501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:39.407501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:05:39.407501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:05:39.407501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:05:39.407501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:05:39.407501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:05:39.407501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:05:39.407501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:05:39.407501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:39.407501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 265/523] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:39.411248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb201c50-c764-4609-8a75-733e93654cee None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:39.417723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1ce8f-9534-4162-82fa-8fde604422c5 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:39.457530 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8501d-5e4c-4679-8adc-132721be1493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:39.458227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8501d-5e4c-4679-8adc-132721be1493 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:39.458578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8501d-5e4c-4679-8adc-132721be1493 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:39.458918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8501d-5e4c-4679-8adc-132721be1493 None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:39.459705 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8501d-5e4c-4679-8adc-132721be1493 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:39.460077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8501d-5e4c-4679-8adc-132721be1493 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:39.461582 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8501d-5e4c-4679-8adc-132721be1493 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:39.483591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 259/524] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:05:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:39.484859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8501d-5e4c-4679-8adc-132721be1493 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:39.521342 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:39.527306 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb201c50-c764-4609-8a75-733e93654cee None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:39.568617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a302bf-31a7-41a5-810c-18c92cb2cd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:39.569293 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a302bf-31a7-41a5-810c-18c92cb2cd4b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:39.569621 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a302bf-31a7-41a5-810c-18c92cb2cd4b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:39.569901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a302bf-31a7-41a5-810c-18c92cb2cd4b None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:39.570756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a302bf-31a7-41a5-810c-18c92cb2cd4b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:39.571113 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a302bf-31a7-41a5-810c-18c92cb2cd4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:39.572943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a302bf-31a7-41a5-810c-18c92cb2cd4b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:39.590759 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8501d-5e4c-4679-8adc-132721be1493 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:39.596996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 266/525] 10.222.0.78 () {56 vars in 1137 bytes} [Mon Sep 19 11:05:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:39.596996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a302bf-31a7-41a5-810c-18c92cb2cd4b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:39.644906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c002a7-a919-476f-a8ac-03141e3917e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:39.645855 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c002a7-a919-476f-a8ac-03141e3917e1 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:39.646312 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c002a7-a919-476f-a8ac-03141e3917e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:39.646761 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c002a7-a919-476f-a8ac-03141e3917e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:39.647972 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c002a7-a919-476f-a8ac-03141e3917e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:39.648489 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c002a7-a919-476f-a8ac-03141e3917e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:39.651683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c002a7-a919-476f-a8ac-03141e3917e1 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:39.676320 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 260/526] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:05:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:39.676320 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c002a7-a919-476f-a8ac-03141e3917e1 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:39.682108 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:39.737282 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a302bf-31a7-41a5-810c-18c92cb2cd4b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:39.776851 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:39.779813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b6f43-0921-4244-aad4-31822842e6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:39.780443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b6f43-0921-4244-aad4-31822842e6bb None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:39.780729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b6f43-0921-4244-aad4-31822842e6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:39.781191 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b6f43-0921-4244-aad4-31822842e6bb None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:39.781774 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b6f43-0921-4244-aad4-31822842e6bb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:39.782061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b6f43-0921-4244-aad4-31822842e6bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:39.783558 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b6f43-0921-4244-aad4-31822842e6bb None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:39.783922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c002a7-a919-476f-a8ac-03141e3917e1 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:39.811390 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 267/527] 10.222.0.78 () {56 vars in 1139 bytes} [Mon Sep 19 11:05:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:39.812822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b6f43-0921-4244-aad4-31822842e6bb None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:39.826338 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06bc4c1-b6c9-4325-8c69-ccbc0e0e2bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:39.826948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06bc4c1-b6c9-4325-8c69-ccbc0e0e2bde None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:39.827254 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06bc4c1-b6c9-4325-8c69-ccbc0e0e2bde None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:39.827531 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06bc4c1-b6c9-4325-8c69-ccbc0e0e2bde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:39.828256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06bc4c1-b6c9-4325-8c69-ccbc0e0e2bde None admin] RBAC: Adding query filter params (role.id=28365abffd3d4ff6874bce2e0c561f1d, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=aba1968529574fe5b4c5d114c874f428) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:39.828568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06bc4c1-b6c9-4325-8c69-ccbc0e0e2bde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:39.829990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06bc4c1-b6c9-4325-8c69-ccbc0e0e2bde None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:39.835625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 261/528] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:05:39 2022] GET /identity/v3/role_assignments?user.id=aba1968529574fe5b4c5d114c874f428&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=28365abffd3d4ff6874bce2e0c561f1d => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:39.836479 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06bc4c1-b6c9-4325-8c69-ccbc0e0e2bde None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:39.939951 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b6f43-0921-4244-aad4-31822842e6bb None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:39.957436 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:39.978297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06bc4c1-b6c9-4325-8c69-ccbc0e0e2bde None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:39.982267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e893544-a155-4a5e-9f70-789bf4154ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:39.983381 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e893544-a155-4a5e-9f70-789bf4154ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:39.983879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e893544-a155-4a5e-9f70-789bf4154ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:39.984333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e893544-a155-4a5e-9f70-789bf4154ac5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:40.026407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a8091-6da6-4f03-a1c1-1c6d83861d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:40.027639 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a8091-6da6-4f03-a1c1-1c6d83861d74 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:40.028091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a8091-6da6-4f03-a1c1-1c6d83861d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:40.028657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a8091-6da6-4f03-a1c1-1c6d83861d74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:40.030072 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a8091-6da6-4f03-a1c1-1c6d83861d74 None admin] RBAC: Adding query filter params (role.id=28365abffd3d4ff6874bce2e0c561f1d, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=1ab94f6e08644fa7a76b9057e46d122d) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:40.030643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a8091-6da6-4f03-a1c1-1c6d83861d74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:40.032314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a8091-6da6-4f03-a1c1-1c6d83861d74 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:40.040602 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 262/529] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:05:39 2022] GET /identity/v3/role_assignments?user.id=1ab94f6e08644fa7a76b9057e46d122d&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=28365abffd3d4ff6874bce2e0c561f1d => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:40.042293 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a8091-6da6-4f03-a1c1-1c6d83861d74 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:40.079491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:40.085661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e893544-a155-4a5e-9f70-789bf4154ac5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:40.088945 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e893544-a155-4a5e-9f70-789bf4154ac5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:40.199698 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a8091-6da6-4f03-a1c1-1c6d83861d74 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:40.224060 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 268/530] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:40.225969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e893544-a155-4a5e-9f70-789bf4154ac5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:40.245409 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:40.246119 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63281979-8d5d-4b8f-9c95-889976481729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:40.246877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63281979-8d5d-4b8f-9c95-889976481729 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:40.247342 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63281979-8d5d-4b8f-9c95-889976481729 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:40.247712 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63281979-8d5d-4b8f-9c95-889976481729 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:40.372401 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63281979-8d5d-4b8f-9c95-889976481729 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:40.373851 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63281979-8d5d-4b8f-9c95-889976481729 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:40.399901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e893544-a155-4a5e-9f70-789bf4154ac5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:40.423947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:40.466296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f3af7-73ce-41a4-8019-93a74cdd2ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:40.468260 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3af7-73ce-41a4-8019-93a74cdd2ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:40.468586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3af7-73ce-41a4-8019-93a74cdd2ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:40.468808 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3af7-73ce-41a4-8019-93a74cdd2ef0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:40.490456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f3af7-73ce-41a4-8019-93a74cdd2ef0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:40.491967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f3af7-73ce-41a4-8019-93a74cdd2ef0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:40.523275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-282f3af7-73ce-41a4-8019-93a74cdd2ef0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:40.523275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:40.523275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:40.523275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:40.523275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:40.523275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:40.523275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:40.523275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:40.523275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:40.523275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:40.523275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:40.523275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:40.523275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:40.523275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:40.523275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:40.523275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:40.524802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:40.524802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:40.524802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:40.524802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:40.524802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:40.524802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:40.524802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:40.524802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:40.524802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:40.524802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:40.524802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:40.524802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:40.524802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:40.524802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:40.524802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:40.524802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:40.524802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:40.525663 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:40.525663 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:40.525663 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:40.531875 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 269/531] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:40.535266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f3af7-73ce-41a4-8019-93a74cdd2ef0 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:40.567707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 263/532] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:40.579910 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63281979-8d5d-4b8f-9c95-889976481729 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:40.638198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:40.720540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f3af7-73ce-41a4-8019-93a74cdd2ef0 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:40.726881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63281979-8d5d-4b8f-9c95-889976481729 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:40.765880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:40.780530 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e82a1-b6d8-4bcb-917c-3019ab801884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:40.781456 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e82a1-b6d8-4bcb-917c-3019ab801884 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:40.781884 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e82a1-b6d8-4bcb-917c-3019ab801884 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:40.782303 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e82a1-b6d8-4bcb-917c-3019ab801884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:40.796406 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e82a1-b6d8-4bcb-917c-3019ab801884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:40.798345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e82a1-b6d8-4bcb-917c-3019ab801884 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:40.812646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd596b-56d5-4bbe-85c9-2e75b5c96dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:40.813275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd596b-56d5-4bbe-85c9-2e75b5c96dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:40.813553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd596b-56d5-4bbe-85c9-2e75b5c96dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:40.813821 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd596b-56d5-4bbe-85c9-2e75b5c96dd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:40.814478 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd596b-56d5-4bbe-85c9-2e75b5c96dd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:40.814780 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd596b-56d5-4bbe-85c9-2e75b5c96dd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:40.816185 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd596b-56d5-4bbe-85c9-2e75b5c96dd5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:40.821616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-853e82a1-b6d8-4bcb-917c-3019ab801884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:40.821616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:40.821616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:40.821616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:40.821616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:40.821616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:40.821616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:40.821616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:40.821616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:40.821616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:40.821616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:40.821616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:40.821616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:40.821616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:40.821616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:40.821616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:40.823448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:40.823448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:40.823448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:40.823448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:40.823448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:40.823448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:40.823448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:40.823448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:40.823448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:40.823448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:40.823448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:40.823448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:40.823448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:40.823448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:40.823448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:40.823448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:40.823448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:40.825051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:40.825051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:40.825051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:40.827516 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 264/533] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:40.833931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e82a1-b6d8-4bcb-917c-3019ab801884 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:40.846631 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 270/534] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:40.854723 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd596b-56d5-4bbe-85c9-2e75b5c96dd5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:40.978384 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd596b-56d5-4bbe-85c9-2e75b5c96dd5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:41.007407 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:41.017649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d66748-2b05-48c2-b87c-dda7755d4685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:41.018598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d66748-2b05-48c2-b87c-dda7755d4685 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:41.019026 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d66748-2b05-48c2-b87c-dda7755d4685 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:41.019469 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d66748-2b05-48c2-b87c-dda7755d4685 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:41.027046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d66748-2b05-48c2-b87c-dda7755d4685 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:41.028927 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d66748-2b05-48c2-b87c-dda7755d4685 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:41.041174 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e82a1-b6d8-4bcb-917c-3019ab801884 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:41.042943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-60d66748-2b05-48c2-b87c-dda7755d4685 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:41.042943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:41.042943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:41.042943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:41.042943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:41.042943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:41.042943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:41.042943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:41.042943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:41.042943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:41.042943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:41.042943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:41.042943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:05:41.042943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:05:41.042943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:05:41.044089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:05:41.044089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:41.044089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:41.044089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:41.044089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:41.044089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:41.044089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:41.044089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:41.044089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:41.044089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:41.044089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:41.044089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:41.044089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:41.044089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:41.044089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:41.044089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:41.044089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:41.045123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:05:41.045123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:05:41.045123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:05:41.045123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:05:41.045123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:05:41.045123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:05:41.045123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:41.045123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:41.047120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 271/535] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:41.059252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d66748-2b05-48c2-b87c-dda7755d4685 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:41.082415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e55ea-ad3b-4586-9fc1-6ea7e8cb0cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:41.083042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e55ea-ad3b-4586-9fc1-6ea7e8cb0cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:41.083345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e55ea-ad3b-4586-9fc1-6ea7e8cb0cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:41.083616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e55ea-ad3b-4586-9fc1-6ea7e8cb0cc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:41.084275 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e55ea-ad3b-4586-9fc1-6ea7e8cb0cc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:41.084573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e55ea-ad3b-4586-9fc1-6ea7e8cb0cc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:41.086000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e55ea-ad3b-4586-9fc1-6ea7e8cb0cc0 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:41.100233 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 265/536] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:41.107114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e55ea-ad3b-4586-9fc1-6ea7e8cb0cc0 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:41.188331 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:41.242029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d66748-2b05-48c2-b87c-dda7755d4685 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:41.281490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a2cdf-bc4e-4c37-a499-01af8c2555f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:41.282103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a2cdf-bc4e-4c37-a499-01af8c2555f0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:41.282381 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a2cdf-bc4e-4c37-a499-01af8c2555f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:41.282650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a2cdf-bc4e-4c37-a499-01af8c2555f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:41.283372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a2cdf-bc4e-4c37-a499-01af8c2555f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:41.283674 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a2cdf-bc4e-4c37-a499-01af8c2555f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:41.285104 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a2cdf-bc4e-4c37-a499-01af8c2555f0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:41.309342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 272/537] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:05:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:41.310599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1a0a2-f4db-40a1-8929-27f6c7192ce2 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:41.310883 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1a0a2-f4db-40a1-8929-27f6c7192ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:41.311157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1a0a2-f4db-40a1-8929-27f6c7192ce2 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:41.312641 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 273/538] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:41.320244 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1a0a2-f4db-40a1-8929-27f6c7192ce2 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:41.323419 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e55ea-ad3b-4586-9fc1-6ea7e8cb0cc0 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:41.360609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc2781-2647-4327-b8e2-ccc3fa9cddc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:41.361662 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc2781-2647-4327-b8e2-ccc3fa9cddc6 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:41.362195 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc2781-2647-4327-b8e2-ccc3fa9cddc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:41.362694 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc2781-2647-4327-b8e2-ccc3fa9cddc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:41.370740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc2781-2647-4327-b8e2-ccc3fa9cddc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:41.372624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc2781-2647-4327-b8e2-ccc3fa9cddc6 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:41.374918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:41.386099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-a0bc2781-2647-4327-b8e2-ccc3fa9cddc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:41.386099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:41.386099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:41.386099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:41.386099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:41.386099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:41.386099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:41.386099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:41.386099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:41.386099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:41.386099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:41.386099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:41.386099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:05:41.386099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:05:41.386099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:05:41.394209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:05:41.394209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:41.394209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:41.394209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:41.394209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:41.394209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:41.394209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:41.394209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:41.394209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:41.394209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:41.394209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:41.394209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:41.394209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:41.394209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:41.394209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:41.394209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:41.394209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:41.395054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:05:41.395054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:05:41.395054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:05:41.395054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:05:41.395054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:05:41.395054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:05:41.395054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:41.395054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:41.395054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 266/539] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:41.395054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c72f2-ce8c-46db-b880-62975361b0c5 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:41.395054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c72f2-ce8c-46db-b880-62975361b0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:41.395657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c72f2-ce8c-46db-b880-62975361b0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:41.413222 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-d92c72f2-ce8c-46db-b880-62975361b0c5 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:41.415968 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d92c72f2-ce8c-46db-b880-62975361b0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:41.443019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1a0a2-f4db-40a1-8929-27f6c7192ce2 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:41.478794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ea839-5a5a-406e-839c-018f88917552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:41.479554 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ea839-5a5a-406e-839c-018f88917552 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:41.479901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ea839-5a5a-406e-839c-018f88917552 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:41.480252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ea839-5a5a-406e-839c-018f88917552 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:41.481060 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ea839-5a5a-406e-839c-018f88917552 None admin] RBAC: Adding query filter params (role.id=28365abffd3d4ff6874bce2e0c561f1d, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=558e3b6dd9754c57ad028aa431a68b80) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:41.481435 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ea839-5a5a-406e-839c-018f88917552 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:41.482952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ea839-5a5a-406e-839c-018f88917552 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:41.489574 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 274/540] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:05:41 2022] GET /identity/v3/role_assignments?user.id=558e3b6dd9754c57ad028aa431a68b80&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=28365abffd3d4ff6874bce2e0c561f1d => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:41.490105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ea839-5a5a-406e-839c-018f88917552 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:41.514312 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 267/541] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:41.633702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ea839-5a5a-406e-839c-018f88917552 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:41.645628 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:41.671880 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf6034-e787-4e93-b44e-abcead9f2d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:41.672510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf6034-e787-4e93-b44e-abcead9f2d80 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:41.672795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf6034-e787-4e93-b44e-abcead9f2d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:41.673072 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf6034-e787-4e93-b44e-abcead9f2d80 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:41.673869 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf6034-e787-4e93-b44e-abcead9f2d80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:41.674180 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf6034-e787-4e93-b44e-abcead9f2d80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:41.675693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf6034-e787-4e93-b44e-abcead9f2d80 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:41.690154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 275/542] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:05:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:41.696042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c72f2-ce8c-46db-b880-62975361b0c5 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:41.724095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:41.814483 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c72f2-ce8c-46db-b880-62975361b0c5 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:41.858645 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58dfc8-6a72-4929-925d-5e78705bc248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:41.859389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58dfc8-6a72-4929-925d-5e78705bc248 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:41.859675 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58dfc8-6a72-4929-925d-5e78705bc248 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:41.859950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58dfc8-6a72-4929-925d-5e78705bc248 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:41.860677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58dfc8-6a72-4929-925d-5e78705bc248 None admin] RBAC: Adding query filter params (role.id=28365abffd3d4ff6874bce2e0c561f1d, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=dbf0049a3e3a442e945f6207477a7fd0) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:41.860984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58dfc8-6a72-4929-925d-5e78705bc248 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:41.862404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58dfc8-6a72-4929-925d-5e78705bc248 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:41.871525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 268/543] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:05:41 2022] GET /identity/v3/role_assignments?user.id=dbf0049a3e3a442e945f6207477a7fd0&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=28365abffd3d4ff6874bce2e0c561f1d => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:41.890814 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf6034-e787-4e93-b44e-abcead9f2d80 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:42.041121 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf6034-e787-4e93-b44e-abcead9f2d80 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:42.091121 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349b477-3486-4602-8ceb-45fe97256a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:42.091729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349b477-3486-4602-8ceb-45fe97256a13 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:42.092009 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349b477-3486-4602-8ceb-45fe97256a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:42.092280 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349b477-3486-4602-8ceb-45fe97256a13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:42.114709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349b477-3486-4602-8ceb-45fe97256a13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:42.123319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-d349b477-3486-4602-8ceb-45fe97256a13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:42.123319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:42.123319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:42.123319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:42.123319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:42.123319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:42.123319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:42.123319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:42.123319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:42.123319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:42.123319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:42.123319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:42.123319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:42.123319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:42.123319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:42.123319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:42.124255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:42.124255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:42.124255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:42.124255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:42.124255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:42.124255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:42.124255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:42.124255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:42.124255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:42.124255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:42.124255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:42.124255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:42.124255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:42.124255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:42.124255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:42.124255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:42.125035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:42.125035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:42.125035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:42.125035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:42.125035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:42.125035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:42.125035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:42.125035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:42.125035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:42.125035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:42.125035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:42.125035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:42.125035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 276/544] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:42.127980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58dfc8-6a72-4929-925d-5e78705bc248 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:42.255197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:42.268420 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58dfc8-6a72-4929-925d-5e78705bc248 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:42.311276 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c05539-2dd6-4575-a235-8666fc07b581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:42.311891 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c05539-2dd6-4575-a235-8666fc07b581 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:42.312169 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c05539-2dd6-4575-a235-8666fc07b581 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:42.312439 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c05539-2dd6-4575-a235-8666fc07b581 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:42.313100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c05539-2dd6-4575-a235-8666fc07b581 None admin] RBAC: Adding query filter params (name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:42.313400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c05539-2dd6-4575-a235-8666fc07b581 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:42.314834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c05539-2dd6-4575-a235-8666fc07b581 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:42.327266 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 269/545] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:05:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:42.332644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349b477-3486-4602-8ceb-45fe97256a13 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:42.402347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:42.506552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349b477-3486-4602-8ceb-45fe97256a13 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:42.566340 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bb4d8-e252-4177-9ef5-4be63fbb5505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:42.566987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bb4d8-e252-4177-9ef5-4be63fbb5505 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:42.567289 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bb4d8-e252-4177-9ef5-4be63fbb5505 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:42.567561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bb4d8-e252-4177-9ef5-4be63fbb5505 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:42.585250 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bb4d8-e252-4177-9ef5-4be63fbb5505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:42.586797 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bb4d8-e252-4177-9ef5-4be63fbb5505 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:42.602111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-2f2bb4d8-e252-4177-9ef5-4be63fbb5505 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:42.602111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:42.602111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:42.602111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:42.602111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:42.602111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:42.602111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:42.602111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:42.602111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:42.602111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:42.602111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:42.602111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:42.602111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:42.602111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:42.602111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:42.602111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:42.603161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:42.603161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:42.603161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:42.603161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:42.603161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:42.603161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:42.603161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:42.603161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:42.603161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:42.603161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:42.603161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:42.603161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:42.603161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:42.603161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:42.603161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:42.603161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:42.603161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:42.604039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:42.604039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:42.604039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:42.604039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 277/546] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:42.612547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c05539-2dd6-4575-a235-8666fc07b581 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:42.651005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e8197-1bb4-40ff-a855-e581bc0ec4ca None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:42.651318 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e8197-1bb4-40ff-a855-e581bc0ec4ca None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:42.651576 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e8197-1bb4-40ff-a855-e581bc0ec4ca None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:42.652972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 278/547] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:05:42.656173 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c724648-d2d0-4801-aab9-4c442a6b411a None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:42.656453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c724648-d2d0-4801-aab9-4c442a6b411a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:42.656724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c724648-d2d0-4801-aab9-4c442a6b411a None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:42.658204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 279/548] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:05:42.663622 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455fe10-a66c-43ff-808f-a8fc56a2b395 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:42.663906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455fe10-a66c-43ff-808f-a8fc56a2b395 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:42.664184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455fe10-a66c-43ff-808f-a8fc56a2b395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:42.686404 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4455fe10-a66c-43ff-808f-a8fc56a2b395 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:42.687656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4455fe10-a66c-43ff-808f-a8fc56a2b395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:42.738914 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c05539-2dd6-4575-a235-8666fc07b581 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:42.770176 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 280/549] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:05:42.771484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94318ff7-ec88-4b29-a207-c28f40dbbcb3 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:42.771763 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94318ff7-ec88-4b29-a207-c28f40dbbcb3 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:42.772033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94318ff7-ec88-4b29-a207-c28f40dbbcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:42.801425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0da647-4653-40ab-9911-e821fe9b8378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:42.802039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0da647-4653-40ab-9911-e821fe9b8378 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:42.802317 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0da647-4653-40ab-9911-e821fe9b8378 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:42.802589 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0da647-4653-40ab-9911-e821fe9b8378 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:42.803254 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0da647-4653-40ab-9911-e821fe9b8378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:42.803558 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0da647-4653-40ab-9911-e821fe9b8378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:42.804922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0da647-4653-40ab-9911-e821fe9b8378 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:42.807642 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-94318ff7-ec88-4b29-a207-c28f40dbbcb3 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:42.808865 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-94318ff7-ec88-4b29-a207-c28f40dbbcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:42.822300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:42.829027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 270/550] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:42.835723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0da647-4653-40ab-9911-e821fe9b8378 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:42.851816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:42.922797 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 281/551] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:42.988430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0da647-4653-40ab-9911-e821fe9b8378 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:43.030569 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59ebb1-a3ef-47a2-8551-c79cff18ed3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:43.031220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59ebb1-a3ef-47a2-8551-c79cff18ed3b None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:43.031502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59ebb1-a3ef-47a2-8551-c79cff18ed3b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:43.031766 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59ebb1-a3ef-47a2-8551-c79cff18ed3b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:43.039145 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59ebb1-a3ef-47a2-8551-c79cff18ed3b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:43.041265 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59ebb1-a3ef-47a2-8551-c79cff18ed3b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:43.053247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-2a59ebb1-a3ef-47a2-8551-c79cff18ed3b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:05:43.053247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:43.053247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:43.053247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:43.053247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:43.053247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:43.053247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:43.053247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:43.053247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:43.053247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:43.053247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:43.053247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:43.053247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:05:43.053247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:05:43.053247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:05:43.053247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:43.054167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:43.055362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:43.055362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:43.055362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:05:43.055362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:05:43.055362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:05:43.055362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:05:43.055362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:05:43.055362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:05:43.055362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:05:43.055362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:43.055362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 271/552] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:05:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:43.061477 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94318ff7-ec88-4b29-a207-c28f40dbbcb3 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:43.124573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59ebb1-a3ef-47a2-8551-c79cff18ed3b None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:43.204771 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:43.237580 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59ebb1-a3ef-47a2-8551-c79cff18ed3b None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:43.273104 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94318ff7-ec88-4b29-a207-c28f40dbbcb3 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:43.273414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3daed-41ee-4cd4-a117-56ec85a83986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:43.274046 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3daed-41ee-4cd4-a117-56ec85a83986 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:43.274858 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3daed-41ee-4cd4-a117-56ec85a83986 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:43.275331 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3daed-41ee-4cd4-a117-56ec85a83986 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:43.286205 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3daed-41ee-4cd4-a117-56ec85a83986 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:43.292583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-bca3daed-41ee-4cd4-a117-56ec85a83986 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:43.292583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:43.292583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:43.292583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:43.292583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:43.292583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:43.292583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:43.292583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:43.292583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:43.292583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:43.292583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:43.292583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:43.292583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:43.292583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:43.292583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:43.292583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:43.293485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:43.293485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:43.293485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:43.293485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:43.293485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:43.293485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:43.293485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:43.293485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:43.293485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:43.293485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:43.293485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:43.293485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:43.293485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:43.293485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:43.293485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:43.293485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:43.294451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:43.294451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:43.294451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:43.294451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:43.294451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:43.294451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:43.294451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:43.294451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:43.294451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:43.294451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:43.294451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:43.294451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:43.294451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 272/553] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:43.301847 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3daed-41ee-4cd4-a117-56ec85a83986 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:43.347890 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110e2f5-07ef-4c6e-be83-19c91069ca99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:43.348560 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110e2f5-07ef-4c6e-be83-19c91069ca99 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:43.348786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110e2f5-07ef-4c6e-be83-19c91069ca99 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:43.349058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110e2f5-07ef-4c6e-be83-19c91069ca99 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:43.349795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110e2f5-07ef-4c6e-be83-19c91069ca99 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:43.350102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110e2f5-07ef-4c6e-be83-19c91069ca99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:43.352161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110e2f5-07ef-4c6e-be83-19c91069ca99 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:43.371945 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:43.376178 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 282/554] 10.222.0.78 () {56 vars in 1135 bytes} [Mon Sep 19 11:05:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:43.377756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110e2f5-07ef-4c6e-be83-19c91069ca99 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:43.433681 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3daed-41ee-4cd4-a117-56ec85a83986 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:43.480938 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:43.488284 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110e2f5-07ef-4c6e-be83-19c91069ca99 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:43.490444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb3242-2ce0-4bd2-ac01-f14f686717f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:43.491915 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb3242-2ce0-4bd2-ac01-f14f686717f0 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:43.493660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb3242-2ce0-4bd2-ac01-f14f686717f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:43.493954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb3242-2ce0-4bd2-ac01-f14f686717f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:43.494621 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb3242-2ce0-4bd2-ac01-f14f686717f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:43.498725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb3242-2ce0-4bd2-ac01-f14f686717f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:43.499928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb3242-2ce0-4bd2-ac01-f14f686717f0 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:43.514389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 273/555] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:05:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:43.514389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb3242-2ce0-4bd2-ac01-f14f686717f0 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:43.542306 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b04bb3-4261-41e2-b204-f074dd2c0940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:43.543478 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b04bb3-4261-41e2-b204-f074dd2c0940 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:43.543907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b04bb3-4261-41e2-b204-f074dd2c0940 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:43.544347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b04bb3-4261-41e2-b204-f074dd2c0940 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:43.558244 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b04bb3-4261-41e2-b204-f074dd2c0940 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:43.563990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-f3b04bb3-4261-41e2-b204-f074dd2c0940 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:43.563990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:43.563990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:43.563990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:43.563990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:43.563990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:43.563990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:43.563990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:43.563990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:43.563990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:43.563990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:43.563990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:43.563990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:43.563990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:43.563990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:43.563990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:43.564909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:43.564909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:43.564909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:43.564909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:43.564909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:43.564909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:43.564909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:43.564909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:43.564909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:43.564909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:43.564909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:43.564909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:43.564909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:43.564909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:43.564909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:43.564909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:43.565818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:43.565818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:43.565818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:43.565818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:43.565818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:43.565818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:43.565818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:43.565818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:43.565818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:43.565818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:43.565818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:43.565818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:43.568912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 283/556] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:43.569988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b04bb3-4261-41e2-b204-f074dd2c0940 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:43.652907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:43.662322 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb3242-2ce0-4bd2-ac01-f14f686717f0 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:43.701240 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b04bb3-4261-41e2-b204-f074dd2c0940 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:43.710993 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd52681-b456-4a5c-80a6-985b41cf0a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:43.712208 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd52681-b456-4a5c-80a6-985b41cf0a0f None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:43.712489 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd52681-b456-4a5c-80a6-985b41cf0a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:43.712759 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd52681-b456-4a5c-80a6-985b41cf0a0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:43.747897 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbdf1b-4360-4c40-820a-ef91af94c59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:43.749852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbdf1b-4360-4c40-820a-ef91af94c59f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:43.750315 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbdf1b-4360-4c40-820a-ef91af94c59f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:43.750738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbdf1b-4360-4c40-820a-ef91af94c59f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:43.766442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbdf1b-4360-4c40-820a-ef91af94c59f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:43.768892 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbdf1b-4360-4c40-820a-ef91af94c59f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:43.779632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:43.790862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd52681-b456-4a5c-80a6-985b41cf0a0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:43.794237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-60cbdf1b-4360-4c40-820a-ef91af94c59f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:43.794237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:43.794237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:43.794237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:43.794237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:43.794237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:43.794237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:43.794237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:43.794237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:43.794237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:43.794237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:43.794237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:43.794237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:43.794237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:43.794237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:43.794237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:43.795408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:43.795408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:43.795408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:43.795408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:43.795408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:43.795408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:43.795408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:43.795408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:43.795408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:43.795408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:43.795408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:43.795408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:43.795408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:43.795408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:43.795408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:43.795408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:43.795408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:43.796487 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:43.796487 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:43.796487 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:43.796657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd52681-b456-4a5c-80a6-985b41cf0a0f None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:43.796736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 284/557] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:43.796736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbdf1b-4360-4c40-820a-ef91af94c59f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:43.911590 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 274/558] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:43.912632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd52681-b456-4a5c-80a6-985b41cf0a0f None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:43.926301 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbdf1b-4360-4c40-820a-ef91af94c59f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:43.941132 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:43.963508 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba0bf4-b427-4ecd-8a9f-0cd355e9794c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:43.964319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba0bf4-b427-4ecd-8a9f-0cd355e9794c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:43.964694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba0bf4-b427-4ecd-8a9f-0cd355e9794c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:43.965067 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba0bf4-b427-4ecd-8a9f-0cd355e9794c None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:43.965859 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ba0bf4-b427-4ecd-8a9f-0cd355e9794c None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:43.966261 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ba0bf4-b427-4ecd-8a9f-0cd355e9794c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:43.967769 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ba0bf4-b427-4ecd-8a9f-0cd355e9794c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:43.977624 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 285/559] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:05:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:43.978673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba0bf4-b427-4ecd-8a9f-0cd355e9794c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:44.044952 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:44.058139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd52681-b456-4a5c-80a6-985b41cf0a0f None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:44.104929 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba0bf4-b427-4ecd-8a9f-0cd355e9794c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:44.109343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496ab0f-91fd-4c5f-90fd-9dcd05735de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:44.110221 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496ab0f-91fd-4c5f-90fd-9dcd05735de6 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:44.110656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496ab0f-91fd-4c5f-90fd-9dcd05735de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:44.111213 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496ab0f-91fd-4c5f-90fd-9dcd05735de6 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:44.112016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496ab0f-91fd-4c5f-90fd-9dcd05735de6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:44.112493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496ab0f-91fd-4c5f-90fd-9dcd05735de6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:44.114672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496ab0f-91fd-4c5f-90fd-9dcd05735de6 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:44.134467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 275/560] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:44.135593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496ab0f-91fd-4c5f-90fd-9dcd05735de6 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:44.145195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20909f8-e8c1-421b-9dd4-38153de3415a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:44.145892 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20909f8-e8c1-421b-9dd4-38153de3415a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:44.146175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20909f8-e8c1-421b-9dd4-38153de3415a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:44.146449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20909f8-e8c1-421b-9dd4-38153de3415a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:44.157832 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20909f8-e8c1-421b-9dd4-38153de3415a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:44.159341 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20909f8-e8c1-421b-9dd4-38153de3415a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:44.174131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-c20909f8-e8c1-421b-9dd4-38153de3415a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:44.174131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:44.174131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:44.174131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:44.174131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:44.174131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:44.174131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:44.174131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:44.174131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:44.174131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:44.174131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:44.174131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:44.174131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:44.174131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:44.174131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:44.174131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:44.175122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:44.175122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:44.175122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:44.175122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:44.175122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:44.175122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:44.175122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:44.175122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:44.175122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:44.175122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:44.175122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:44.175122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:44.175122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:44.175122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:44.175122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:44.175122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:44.175122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:44.176774 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:44.176774 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:44.176774 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:44.176774 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 286/561] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:44.176774 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614d8d7-0449-42ab-aecd-1e451d70bc71 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:44.177072 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614d8d7-0449-42ab-aecd-1e451d70bc71 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:44.177300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614d8d7-0449-42ab-aecd-1e451d70bc71 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:44.178675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 287/562] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:44.183683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614d8d7-0449-42ab-aecd-1e451d70bc71 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:44.215594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:44.257975 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496ab0f-91fd-4c5f-90fd-9dcd05735de6 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:44.291258 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614d8d7-0449-42ab-aecd-1e451d70bc71 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:44.293037 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b061d6b-eac1-42cd-b685-f746a07cb455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:44.293728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b061d6b-eac1-42cd-b685-f746a07cb455 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:44.294005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b061d6b-eac1-42cd-b685-f746a07cb455 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:44.294327 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b061d6b-eac1-42cd-b685-f746a07cb455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:44.306005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b061d6b-eac1-42cd-b685-f746a07cb455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:44.307425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b061d6b-eac1-42cd-b685-f746a07cb455 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:44.322443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-6b061d6b-eac1-42cd-b685-f746a07cb455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:44.322443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:44.322443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:44.322443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:44.322443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:44.322443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:44.322443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:44.322443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:44.322443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:44.322443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:44.322443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:44.322443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:44.322443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:44.322443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:44.322443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:44.322443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:44.323424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:44.323424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:44.323424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:44.323424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:44.323424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:44.323424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:44.323424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:44.323424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:44.323424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:44.323424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:44.323424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:44.323424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:44.323424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:44.323424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:44.323424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:44.323424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:44.323424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:44.324496 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:44.324496 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:44.324496 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:44.328308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 276/563] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:44.329143 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b061d6b-eac1-42cd-b685-f746a07cb455 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:44.332048 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175fe61-2dcd-4acf-bd33-20b34e6297e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:44.332873 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175fe61-2dcd-4acf-bd33-20b34e6297e5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:44.333153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175fe61-2dcd-4acf-bd33-20b34e6297e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:44.333430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175fe61-2dcd-4acf-bd33-20b34e6297e5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:44.333713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:44.341412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175fe61-2dcd-4acf-bd33-20b34e6297e5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:44.342872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175fe61-2dcd-4acf-bd33-20b34e6297e5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:44.356015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-d175fe61-2dcd-4acf-bd33-20b34e6297e5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:05:44.356015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:44.356015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:44.356015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:44.356015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:44.356015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:44.356015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:44.356015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:44.356015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:44.356015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:44.356015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:44.356015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:44.356015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:05:44.356015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:05:44.356015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:05:44.356015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:44.357494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:44.358835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:44.358835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:44.358835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:05:44.358835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:05:44.358835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:05:44.358835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:05:44.358835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:05:44.358835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:05:44.358835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:05:44.358835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:44.358835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 288/564] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:44.358835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5bc52-463f-4e31-9314-43ceec01dad1 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:44.358835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5bc52-463f-4e31-9314-43ceec01dad1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:44.359583 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5bc52-463f-4e31-9314-43ceec01dad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:44.377435 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8ed5bc52-463f-4e31-9314-43ceec01dad1 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:44.378647 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8ed5bc52-463f-4e31-9314-43ceec01dad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:44.448952 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b061d6b-eac1-42cd-b685-f746a07cb455 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:44.455445 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 289/565] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:44.456963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5bc52-463f-4e31-9314-43ceec01dad1 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:44.488519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:44.489643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a4b54-2b8f-4907-bc57-a25c5ee93053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:44.490224 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a4b54-2b8f-4907-bc57-a25c5ee93053 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:44.490506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a4b54-2b8f-4907-bc57-a25c5ee93053 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:44.490777 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a4b54-2b8f-4907-bc57-a25c5ee93053 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:44.491463 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a4b54-2b8f-4907-bc57-a25c5ee93053 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:44.491765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a4b54-2b8f-4907-bc57-a25c5ee93053 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:44.493953 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a4b54-2b8f-4907-bc57-a25c5ee93053 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:44.507388 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 277/566] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:44.508261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a4b54-2b8f-4907-bc57-a25c5ee93053 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:44.574115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:44.588839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5bc52-463f-4e31-9314-43ceec01dad1 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:44.615082 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a4b54-2b8f-4907-bc57-a25c5ee93053 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:44.625308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1607c-4df5-41ad-ade8-329376ed8373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:44.625923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1607c-4df5-41ad-ade8-329376ed8373 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:44.626208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1607c-4df5-41ad-ade8-329376ed8373 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:44.626481 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1607c-4df5-41ad-ade8-329376ed8373 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:44.627515 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1607c-4df5-41ad-ade8-329376ed8373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:44.627721 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1607c-4df5-41ad-ade8-329376ed8373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:44.629089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1607c-4df5-41ad-ade8-329376ed8373 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:44.645988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 290/567] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:44.646866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1607c-4df5-41ad-ade8-329376ed8373 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:44.652724 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757699bd-2b11-474f-9d1a-b07683940966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:44.653340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757699bd-2b11-474f-9d1a-b07683940966 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:44.653622 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757699bd-2b11-474f-9d1a-b07683940966 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:44.653895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757699bd-2b11-474f-9d1a-b07683940966 None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:44.654626 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757699bd-2b11-474f-9d1a-b07683940966 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:44.654931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757699bd-2b11-474f-9d1a-b07683940966 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:44.656819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757699bd-2b11-474f-9d1a-b07683940966 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:44.700033 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 278/568] 10.222.0.78 () {56 vars in 1137 bytes} [Mon Sep 19 11:05:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:44.700033 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca025f1-a407-4bd7-bcfa-600bc478179c None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:44.700033 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca025f1-a407-4bd7-bcfa-600bc478179c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:44.700033 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca025f1-a407-4bd7-bcfa-600bc478179c None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:44.700033 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 279/569] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:44.700033 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca025f1-a407-4bd7-bcfa-600bc478179c None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:44.735429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:44.759508 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1607c-4df5-41ad-ade8-329376ed8373 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:44.795349 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07365c9-3857-4653-8010-54b5ed53dac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:44.795990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07365c9-3857-4653-8010-54b5ed53dac4 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:44.796271 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07365c9-3857-4653-8010-54b5ed53dac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:44.796544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07365c9-3857-4653-8010-54b5ed53dac4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:44.803193 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07365c9-3857-4653-8010-54b5ed53dac4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:44.804660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07365c9-3857-4653-8010-54b5ed53dac4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:44.804899 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca025f1-a407-4bd7-bcfa-600bc478179c None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:44.815904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-f07365c9-3857-4653-8010-54b5ed53dac4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:44.815904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:44.815904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:44.815904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:44.815904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:44.815904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:44.815904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:44.815904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:44.815904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:44.815904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:44.815904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:44.815904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:44.815904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:05:44.815904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:05:44.815904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:05:44.816964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:05:44.816964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:44.816964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:44.816964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:44.816964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:44.816964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:44.816964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:44.816964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:44.816964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:44.816964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:44.816964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:44.816964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:44.816964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:44.816964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:44.816964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:44.816964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:44.816964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:44.818059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:05:44.818059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:05:44.818059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:05:44.818059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:05:44.818059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:05:44.818059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:05:44.818059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:44.818059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:44.818059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 291/570] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:44.818059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07365c9-3857-4653-8010-54b5ed53dac4 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:44.830843 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:44.843286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ad755-77c6-4932-a3f2-2d6b57704d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:44.851880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ad755-77c6-4932-a3f2-2d6b57704d15 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:44.851880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ad755-77c6-4932-a3f2-2d6b57704d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:44.851880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ad755-77c6-4932-a3f2-2d6b57704d15 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:44.851880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ad755-77c6-4932-a3f2-2d6b57704d15 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:44.853659 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ad755-77c6-4932-a3f2-2d6b57704d15 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:44.865244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-0e5ad755-77c6-4932-a3f2-2d6b57704d15 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:05:44.865244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:44.865244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:44.865244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:44.865244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:44.865244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:44.865244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:44.865244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:44.865244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:44.865244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:44.865244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:44.865244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:44.865244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:05:44.865244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:05:44.865244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:05:44.865244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:44.866380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:44.867603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:44.867603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:44.867603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:05:44.867603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:05:44.867603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:05:44.867603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:05:44.867603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:05:44.867603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:05:44.867603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:05:44.867603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:44.867603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 280/571] 10.222.0.78 () {56 vars in 1093 bytes} [Mon Sep 19 11:05:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:44.867603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501c157-a732-4c17-b829-f264d20de25d None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:44.868387 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501c157-a732-4c17-b829-f264d20de25d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:44.868387 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501c157-a732-4c17-b829-f264d20de25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:44.886291 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-d501c157-a732-4c17-b829-f264d20de25d None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:44.887535 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d501c157-a732-4c17-b829-f264d20de25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:44.925466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07365c9-3857-4653-8010-54b5ed53dac4 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:44.954784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 281/572] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:44.955664 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501c157-a732-4c17-b829-f264d20de25d None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:44.966289 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ececf5-c985-4297-84ce-6b8f653fb0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:44.966935 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ececf5-c985-4297-84ce-6b8f653fb0df None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:44.967271 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ececf5-c985-4297-84ce-6b8f653fb0df None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:44.967537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ececf5-c985-4297-84ce-6b8f653fb0df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:44.974703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:44.978509 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ececf5-c985-4297-84ce-6b8f653fb0df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:44.980008 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ececf5-c985-4297-84ce-6b8f653fb0df None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:44.994510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-f7ececf5-c985-4297-84ce-6b8f653fb0df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:44.994510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:44.994510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:44.994510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:44.994510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:44.994510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:44.994510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:44.994510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:44.994510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:44.994510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:44.994510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:44.994510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:44.994510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:44.994510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:44.994510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:44.994510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:44.995715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:44.995715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:44.995715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:44.995715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:44.995715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:44.995715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:44.995715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:44.995715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:44.995715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:44.995715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:44.995715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:44.995715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:44.995715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:44.995715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:44.995715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:44.995715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:44.995715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:44.996839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:44.996839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:44.996839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:44.996839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 292/573] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:44.996839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ececf5-c985-4297-84ce-6b8f653fb0df None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:45.070243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501c157-a732-4c17-b829-f264d20de25d None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:45.084185 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:45.105556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5323d-c080-4a06-8c71-9922e69bd836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:45.107947 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ececf5-c985-4297-84ce-6b8f653fb0df None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:45.114271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5323d-c080-4a06-8c71-9922e69bd836 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:45.114558 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5323d-c080-4a06-8c71-9922e69bd836 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:45.114834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5323d-c080-4a06-8c71-9922e69bd836 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:45.126848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5323d-c080-4a06-8c71-9922e69bd836 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:45.133036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-f2d5323d-c080-4a06-8c71-9922e69bd836 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:45.133036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:45.133036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:45.133036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:45.133036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:45.133036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:45.133036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:45.133036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:45.133036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:45.133036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:45.133036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:45.133036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:45.133036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:45.133036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:45.133036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:45.133036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:45.134151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:45.134151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:45.134151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:45.134151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:45.134151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:45.134151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:45.134151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:45.134151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:45.134151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:45.134151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:45.134151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:45.134151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:45.134151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:45.134151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:45.134151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:45.134151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:45.140256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:45.140256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:45.140256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:45.140256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:45.140256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:45.140256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:45.140256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:45.140256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:45.140256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:45.140256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:45.140256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:45.140256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:45.140256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 282/574] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:45.140256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5323d-c080-4a06-8c71-9922e69bd836 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:45.151302 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d80300-c091-40de-92da-6d42d769bc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:45.152308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d80300-c091-40de-92da-6d42d769bc46 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:45.152595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d80300-c091-40de-92da-6d42d769bc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:45.153060 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d80300-c091-40de-92da-6d42d769bc46 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:45.153764 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d80300-c091-40de-92da-6d42d769bc46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:45.154124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d80300-c091-40de-92da-6d42d769bc46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:45.155633 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d80300-c091-40de-92da-6d42d769bc46 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:45.168377 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 293/575] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:05:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:45.169204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d80300-c091-40de-92da-6d42d769bc46 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:45.264643 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:45.315578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5323d-c080-4a06-8c71-9922e69bd836 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:45.316839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d80300-c091-40de-92da-6d42d769bc46 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:45.349043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558c373-0457-450f-ae93-7f1ac392508e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:45.349567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558c373-0457-450f-ae93-7f1ac392508e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:45.349849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558c373-0457-450f-ae93-7f1ac392508e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:45.350129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558c373-0457-450f-ae93-7f1ac392508e None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:45.350850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558c373-0457-450f-ae93-7f1ac392508e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:45.351155 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558c373-0457-450f-ae93-7f1ac392508e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:45.353134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558c373-0457-450f-ae93-7f1ac392508e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:45.353684 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807bb69-3756-4dad-a5bc-233bdd1479a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:45.354298 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807bb69-3756-4dad-a5bc-233bdd1479a2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:45.354576 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807bb69-3756-4dad-a5bc-233bdd1479a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:45.354853 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807bb69-3756-4dad-a5bc-233bdd1479a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:45.355525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807bb69-3756-4dad-a5bc-233bdd1479a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:45.355837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807bb69-3756-4dad-a5bc-233bdd1479a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:45.357206 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807bb69-3756-4dad-a5bc-233bdd1479a2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:45.369929 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 294/576] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:45.370762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807bb69-3756-4dad-a5bc-233bdd1479a2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:45.375351 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 283/577] 10.222.0.78 () {56 vars in 1133 bytes} [Mon Sep 19 11:05:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:45.376304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558c373-0457-450f-ae93-7f1ac392508e None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:45.406088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:45.474569 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807bb69-3756-4dad-a5bc-233bdd1479a2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:45.476332 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558c373-0457-450f-ae93-7f1ac392508e None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:45.484568 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:45.508896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fcff7-f0d2-421e-ae6f-2e40646e65e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:45.509525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fcff7-f0d2-421e-ae6f-2e40646e65e3 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:45.509802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fcff7-f0d2-421e-ae6f-2e40646e65e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:45.510069 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fcff7-f0d2-421e-ae6f-2e40646e65e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:45.510555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458c34e-01ce-47b2-ac90-aae5854b6c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:45.510723 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07fcff7-f0d2-421e-ae6f-2e40646e65e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:45.511069 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07fcff7-f0d2-421e-ae6f-2e40646e65e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:45.511183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c34e-01ce-47b2-ac90-aae5854b6c8d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:45.511466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c34e-01ce-47b2-ac90-aae5854b6c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:45.511737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c34e-01ce-47b2-ac90-aae5854b6c8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:45.512465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07fcff7-f0d2-421e-ae6f-2e40646e65e3 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:45.512556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458c34e-01ce-47b2-ac90-aae5854b6c8d None admin] RBAC: Adding query filter params (role.id=28365abffd3d4ff6874bce2e0c561f1d, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=cf462b2aa3fc4e2ca58851549cf0c644) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:45.512803 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458c34e-01ce-47b2-ac90-aae5854b6c8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:45.514225 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458c34e-01ce-47b2-ac90-aae5854b6c8d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:45.519568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 284/578] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:05:45 2022] GET /identity/v3/role_assignments?user.id=cf462b2aa3fc4e2ca58851549cf0c644&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=28365abffd3d4ff6874bce2e0c561f1d => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:45.520361 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458c34e-01ce-47b2-ac90-aae5854b6c8d None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:45.525316 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 295/579] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:05:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:45.526304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fcff7-f0d2-421e-ae6f-2e40646e65e3 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:45.646917 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:45.651661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fcff7-f0d2-421e-ae6f-2e40646e65e3 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:45.654982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458c34e-01ce-47b2-ac90-aae5854b6c8d None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:45.692570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd20ac3-258b-428f-a065-bc4fb8cf4629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:45.693212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd20ac3-258b-428f-a065-bc4fb8cf4629 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:45.693490 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd20ac3-258b-428f-a065-bc4fb8cf4629 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:45.693758 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd20ac3-258b-428f-a065-bc4fb8cf4629 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:45.695938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb38bbb-c783-444f-b451-ca97fc69e5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:45.696744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb38bbb-c783-444f-b451-ca97fc69e5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:45.697025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb38bbb-c783-444f-b451-ca97fc69e5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:45.698155 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb38bbb-c783-444f-b451-ca97fc69e5e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:45.705420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb38bbb-c783-444f-b451-ca97fc69e5e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:45.706971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb38bbb-c783-444f-b451-ca97fc69e5e7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:45.707288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd20ac3-258b-428f-a065-bc4fb8cf4629 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:45.716336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-4dd20ac3-258b-428f-a065-bc4fb8cf4629 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:45.716336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:45.716336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:45.716336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:45.716336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:45.716336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:45.716336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:45.716336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:45.716336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:45.716336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:45.716336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:45.716336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:45.716336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:45.716336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:45.716336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:45.716336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:45.717481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:45.717481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:45.717481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:45.717481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:45.717481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:45.717481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:45.717481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:45.717481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:45.717481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:45.717481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:45.717481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:45.717481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:45.717481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:45.717481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:45.717481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:45.717481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:45.718525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:45.718525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:45.718525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:45.718525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:45.718525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:45.718525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:45.718525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:45.718525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:45.718525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:45.718525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:45.718525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:45.718525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:45.719608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-5fb38bbb-c783-444f-b451-ca97fc69e5e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:45.719608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:45.719608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:45.719608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:45.719608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:45.719608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:45.719608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:45.719608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:45.719608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:45.719608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:45.719608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:45.719608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:45.719608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:05:45.719608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:05:45.719608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:05:45.720675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:05:45.720675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:45.720675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:45.720675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:45.720675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:45.720675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:45.720675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:45.720675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:45.720675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:45.720675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:45.720675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:45.720675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:45.720675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:45.720675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:45.720675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:45.720675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:45.720675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:45.721765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:05:45.721765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:05:45.721765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:05:45.721765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:05:45.721765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:05:45.721765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:05:45.721765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:45.721765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:45.727212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 285/580] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:45.727212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd20ac3-258b-428f-a065-bc4fb8cf4629 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:45.727954 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 296/581] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:45.727954 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb38bbb-c783-444f-b451-ca97fc69e5e7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:45.761021 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:45.831322 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd20ac3-258b-428f-a065-bc4fb8cf4629 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:45.831536 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb38bbb-c783-444f-b451-ca97fc69e5e7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:45.866394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac4891-2f82-4db5-9380-87b14f588dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:45.866777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfdf28-0905-41d2-849b-4de626e2045d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:45.867207 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac4891-2f82-4db5-9380-87b14f588dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:45.867557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac4891-2f82-4db5-9380-87b14f588dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:45.867651 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfdf28-0905-41d2-849b-4de626e2045d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:45.867998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfdf28-0905-41d2-849b-4de626e2045d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:45.868090 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac4891-2f82-4db5-9380-87b14f588dd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:45.868420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfdf28-0905-41d2-849b-4de626e2045d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:45.879567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac4891-2f82-4db5-9380-87b14f588dd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:45.879974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfdf28-0905-41d2-849b-4de626e2045d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:45.881239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac4891-2f82-4db5-9380-87b14f588dd6 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:45.881550 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfdf28-0905-41d2-849b-4de626e2045d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:45.896895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-2bac4891-2f82-4db5-9380-87b14f588dd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:45.896895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:45.896895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:45.896895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:45.896895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:45.896895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:45.896895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:45.896895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:45.896895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:45.896895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:45.896895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:45.896895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:45.896895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:45.896895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:45.896895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:45.896895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:45.898112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:45.898112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:45.898112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:45.898112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:45.898112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:45.898112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:45.898112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:45.898112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:45.898112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:45.898112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:45.898112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:45.898112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:45.898112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:45.898112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:45.898112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:45.898112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:45.898112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:45.899831 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:45.899831 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:45.899831 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:45.900053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-efdfdf28-0905-41d2-849b-4de626e2045d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:45.900053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:45.900053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:45.900053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:45.900053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:45.900053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:45.900053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:45.900053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:45.900053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:45.900053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:45.900053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:45.900053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:45.900053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:45.900053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:45.900053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:45.900053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:45.905943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:45.905943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:45.905943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:45.905943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:45.905943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:45.905943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:45.905943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:45.905943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:45.905943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:45.905943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:45.905943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:45.905943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:45.905943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:45.905943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:45.905943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:45.905943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:45.905943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:45.907073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:45.907073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:45.907073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:45.907305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 286/582] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:45.907305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac4891-2f82-4db5-9380-87b14f588dd6 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:45.907472 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 297/583] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:45.907647 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfdf28-0905-41d2-849b-4de626e2045d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:45.914970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:45.992681 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:46.013305 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfdf28-0905-41d2-849b-4de626e2045d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:46.016267 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac4891-2f82-4db5-9380-87b14f588dd6 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:46.050136 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07726a3c-0bd5-40ad-9885-fa1e476839ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:46.050750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07726a3c-0bd5-40ad-9885-fa1e476839ec None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:46.051027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07726a3c-0bd5-40ad-9885-fa1e476839ec None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:46.051325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07726a3c-0bd5-40ad-9885-fa1e476839ec None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:46.051671 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b79347-124f-4858-9e9f-53ccf56c6267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:46.052020 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07726a3c-0bd5-40ad-9885-fa1e476839ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:46.052324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07726a3c-0bd5-40ad-9885-fa1e476839ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:46.052778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b79347-124f-4858-9e9f-53ccf56c6267 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:46.053204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b79347-124f-4858-9e9f-53ccf56c6267 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:46.053633 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b79347-124f-4858-9e9f-53ccf56c6267 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:46.053759 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07726a3c-0bd5-40ad-9885-fa1e476839ec None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:46.054564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b79347-124f-4858-9e9f-53ccf56c6267 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:46.055015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b79347-124f-4858-9e9f-53ccf56c6267 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:46.056742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b79347-124f-4858-9e9f-53ccf56c6267 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:46.065594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 298/584] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:05:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:46.066223 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 287/585] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:05:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:46.066766 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b79347-124f-4858-9e9f-53ccf56c6267 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:46.067098 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07726a3c-0bd5-40ad-9885-fa1e476839ec None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:46.139279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:46.179615 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07726a3c-0bd5-40ad-9885-fa1e476839ec None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:46.193455 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b79347-124f-4858-9e9f-53ccf56c6267 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:46.227433 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229f017-49f9-4311-bc3c-c8645fa9d314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:46.228049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229f017-49f9-4311-bc3c-c8645fa9d314 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:46.228514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229f017-49f9-4311-bc3c-c8645fa9d314 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:46.228634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229f017-49f9-4311-bc3c-c8645fa9d314 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:46.229228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229f017-49f9-4311-bc3c-c8645fa9d314 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:46.229531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229f017-49f9-4311-bc3c-c8645fa9d314 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:46.230918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229f017-49f9-4311-bc3c-c8645fa9d314 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:46.244142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 299/586] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:46.244786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229f017-49f9-4311-bc3c-c8645fa9d314 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:46.249322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:46.260322 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82262a1-8d65-45ff-949f-3ce9971ad92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:46.262725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82262a1-8d65-45ff-949f-3ce9971ad92e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:46.262725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82262a1-8d65-45ff-949f-3ce9971ad92e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:46.262725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82262a1-8d65-45ff-949f-3ce9971ad92e None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:46.262725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82262a1-8d65-45ff-949f-3ce9971ad92e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:46.262725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82262a1-8d65-45ff-949f-3ce9971ad92e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:46.263848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82262a1-8d65-45ff-949f-3ce9971ad92e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:46.280834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 288/587] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:46.280834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82262a1-8d65-45ff-949f-3ce9971ad92e None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:46.357592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229f017-49f9-4311-bc3c-c8645fa9d314 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:46.387979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82262a1-8d65-45ff-949f-3ce9971ad92e None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:46.405322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad651c8-ecfa-439e-a909-a656a04197ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:46.406079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad651c8-ecfa-439e-a909-a656a04197ab None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:46.406415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad651c8-ecfa-439e-a909-a656a04197ab None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:46.406857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad651c8-ecfa-439e-a909-a656a04197ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:46.419687 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad651c8-ecfa-439e-a909-a656a04197ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:46.420950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:46.421255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad651c8-ecfa-439e-a909-a656a04197ab None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:46.436467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd9b97-f4a0-418f-9cb9-d613d94bde03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:46.436467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd9b97-f4a0-418f-9cb9-d613d94bde03 None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:46.436467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd9b97-f4a0-418f-9cb9-d613d94bde03 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:46.436467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd9b97-f4a0-418f-9cb9-d613d94bde03 None admin] PATH_INFO: `/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/1ab94f6e08644fa7a76b9057e46d122d/roles/28365abffd3d4ff6874bce2e0c561f1d` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:46.437743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-4ad651c8-ecfa-439e-a909-a656a04197ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:46.437743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:46.437743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:46.437743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:46.437743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:46.437743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:46.437743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:46.437743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:46.437743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:46.437743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:46.437743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:46.437743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:46.437743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:46.437743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:46.437743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:46.437743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:46.438902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:46.438902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:46.438902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:46.438902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:46.438902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:46.438902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:46.438902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:46.438902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:46.438902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:46.438902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:46.438902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:46.438902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:46.438902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:46.438902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:46.438902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:46.438902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:46.438902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:46.440157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:46.440157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:46.440157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:46.443686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 300/588] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:46.444289 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad651c8-ecfa-439e-a909-a656a04197ab None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:46.451611 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd9b97-f4a0-418f-9cb9-d613d94bde03 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8563e86b7c94ff29e39fb20a7969551, user_id=1ab94f6e08644fa7a76b9057e46d122d, role_id=28365abffd3d4ff6874bce2e0c561f1d)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:46.453198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd9b97-f4a0-418f-9cb9-d613d94bde03 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:46.502486 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:46.525270 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 289/589] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:05:46 2022] PUT /identity/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/1ab94f6e08644fa7a76b9057e46d122d/roles/28365abffd3d4ff6874bce2e0c561f1d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:05:46.536224 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd9b97-f4a0-418f-9cb9-d613d94bde03 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:46.571464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad651c8-ecfa-439e-a909-a656a04197ab None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:46.612725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77736f2f-b65f-4639-8768-be3f5b39f204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:46.613425 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77736f2f-b65f-4639-8768-be3f5b39f204 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:46.613762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77736f2f-b65f-4639-8768-be3f5b39f204 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:46.614039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77736f2f-b65f-4639-8768-be3f5b39f204 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:46.622692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77736f2f-b65f-4639-8768-be3f5b39f204 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:46.624232 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77736f2f-b65f-4639-8768-be3f5b39f204 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:46.637594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-77736f2f-b65f-4639-8768-be3f5b39f204 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:46.637594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:46.637594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:46.637594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:46.637594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:46.637594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:46.637594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:46.637594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:46.637594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:46.637594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:46.637594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:46.637594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:46.637594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:05:46.637594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:05:46.637594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:05:46.638551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:05:46.638551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:46.638551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:46.638551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:46.638551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:46.638551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:46.638551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:46.638551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:46.638551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:46.638551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:46.638551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:46.638551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:46.638551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:46.638551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:46.638551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:46.638551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:46.638551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:46.639482 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:05:46.639482 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:05:46.639482 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:05:46.639482 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:05:46.639482 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:05:46.639482 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:05:46.639482 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:46.639482 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:46.639482 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 301/590] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:46.639963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77736f2f-b65f-4639-8768-be3f5b39f204 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:46.699721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd9b97-f4a0-418f-9cb9-d613d94bde03 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:46.746441 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2933ea-5681-4c9a-89e4-fb9b3f203e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:46.747650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2933ea-5681-4c9a-89e4-fb9b3f203e51 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:46.748165 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2933ea-5681-4c9a-89e4-fb9b3f203e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:46.748769 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2933ea-5681-4c9a-89e4-fb9b3f203e51 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:46.758134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2933ea-5681-4c9a-89e4-fb9b3f203e51 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:46.759677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2933ea-5681-4c9a-89e4-fb9b3f203e51 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:46.774151 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77736f2f-b65f-4639-8768-be3f5b39f204 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:46.785995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-2e2933ea-5681-4c9a-89e4-fb9b3f203e51 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:05:46.785995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:46.785995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:46.785995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:46.785995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:46.785995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:46.785995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:46.785995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:46.785995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:46.785995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:46.785995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:46.785995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:46.785995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:05:46.785995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:05:46.785995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:05:46.785995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:46.787140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:46.788296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:46.788296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:46.788296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:05:46.788296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:05:46.788296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:05:46.788296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:05:46.788296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:05:46.788296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:05:46.788296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:05:46.788296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:46.791197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 290/591] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:46.792067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2933ea-5681-4c9a-89e4-fb9b3f203e51 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:46.797043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:46.815989 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451e43a-196d-4ed7-9fa5-c36c10ad6b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:46.816966 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451e43a-196d-4ed7-9fa5-c36c10ad6b19 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:46.817465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451e43a-196d-4ed7-9fa5-c36c10ad6b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:46.817879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451e43a-196d-4ed7-9fa5-c36c10ad6b19 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:46.818667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451e43a-196d-4ed7-9fa5-c36c10ad6b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:46.819111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451e43a-196d-4ed7-9fa5-c36c10ad6b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:46.820829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451e43a-196d-4ed7-9fa5-c36c10ad6b19 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:46.834798 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 302/592] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:46.836056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451e43a-196d-4ed7-9fa5-c36c10ad6b19 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:46.836514 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:46.935765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2933ea-5681-4c9a-89e4-fb9b3f203e51 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:46.951936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451e43a-196d-4ed7-9fa5-c36c10ad6b19 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:46.978812 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb10e94-ee6e-4363-a22b-e1b807dac3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:46.979599 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb10e94-ee6e-4363-a22b-e1b807dac3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:46.980044 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb10e94-ee6e-4363-a22b-e1b807dac3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:46.980390 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb10e94-ee6e-4363-a22b-e1b807dac3a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:46.981230 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb10e94-ee6e-4363-a22b-e1b807dac3a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:46.981608 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb10e94-ee6e-4363-a22b-e1b807dac3a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:46.983273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb10e94-ee6e-4363-a22b-e1b807dac3a2 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:46.988534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db510bd3-4ce1-4b63-bbf2-2ee723f21337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:46.989138 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db510bd3-4ce1-4b63-bbf2-2ee723f21337 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:46.989414 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db510bd3-4ce1-4b63-bbf2-2ee723f21337 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:46.989683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db510bd3-4ce1-4b63-bbf2-2ee723f21337 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:46.990402 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db510bd3-4ce1-4b63-bbf2-2ee723f21337 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:46.990804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db510bd3-4ce1-4b63-bbf2-2ee723f21337 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:46.992356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db510bd3-4ce1-4b63-bbf2-2ee723f21337 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:46.996478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 291/593] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:05:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:46.997324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb10e94-ee6e-4363-a22b-e1b807dac3a2 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:47.021110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 303/594] 10.222.0.78 () {56 vars in 1137 bytes} [Mon Sep 19 11:05:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:47.022021 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db510bd3-4ce1-4b63-bbf2-2ee723f21337 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:47.059102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:47.105318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:47.111641 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb10e94-ee6e-4363-a22b-e1b807dac3a2 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:47.135209 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db510bd3-4ce1-4b63-bbf2-2ee723f21337 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:47.149621 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d67ae-318b-43c5-99c1-7a94dfe3032c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:47.150355 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d67ae-318b-43c5-99c1-7a94dfe3032c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:47.150697 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d67ae-318b-43c5-99c1-7a94dfe3032c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:47.151048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d67ae-318b-43c5-99c1-7a94dfe3032c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:47.158977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d67ae-318b-43c5-99c1-7a94dfe3032c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:47.161635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d67ae-318b-43c5-99c1-7a94dfe3032c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:47.174592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-be9d67ae-318b-43c5-99c1-7a94dfe3032c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:05:47.174592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:47.174592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:47.174592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:47.174592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:47.174592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:47.174592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:47.174592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:47.174592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:47.174592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:47.174592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:47.174592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:47.174592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:05:47.174592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:05:47.174592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:05:47.174592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:47.175783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:47.176918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:47.176918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:47.176918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:05:47.176918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:05:47.176918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:05:47.176918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:05:47.176918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:05:47.176918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:05:47.176918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:05:47.176918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:47.179357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 292/595] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:47.180418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d67ae-318b-43c5-99c1-7a94dfe3032c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:47.190815 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e76b7-9511-49cc-8d9d-edc3a86dad3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:47.191548 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e76b7-9511-49cc-8d9d-edc3a86dad3d None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:47.191840 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e76b7-9511-49cc-8d9d-edc3a86dad3d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:47.192123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e76b7-9511-49cc-8d9d-edc3a86dad3d None admin] PATH_INFO: `/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/aba1968529574fe5b4c5d114c874f428/roles/28365abffd3d4ff6874bce2e0c561f1d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:47.207690 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e76b7-9511-49cc-8d9d-edc3a86dad3d None admin] RBAC: Authorizing `identity:create_grant(project_id=a8563e86b7c94ff29e39fb20a7969551, user_id=aba1968529574fe5b4c5d114c874f428, role_id=28365abffd3d4ff6874bce2e0c561f1d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:47.209058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e76b7-9511-49cc-8d9d-edc3a86dad3d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:47.283405 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 304/596] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:05:47 2022] PUT /identity/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/aba1968529574fe5b4c5d114c874f428/roles/28365abffd3d4ff6874bce2e0c561f1d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:05:47.284673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e76b7-9511-49cc-8d9d-edc3a86dad3d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:47.356804 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d67ae-318b-43c5-99c1-7a94dfe3032c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:47.404062 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:47.422049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e76b7-9511-49cc-8d9d-edc3a86dad3d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:47.444815 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373aa19f-3796-4efe-9ed5-59fb491c294c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:47.446953 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373aa19f-3796-4efe-9ed5-59fb491c294c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:47.455201 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373aa19f-3796-4efe-9ed5-59fb491c294c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:47.456004 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373aa19f-3796-4efe-9ed5-59fb491c294c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:47.466406 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0082b37-864b-4c10-bf7b-fa323987001d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:47.467014 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0082b37-864b-4c10-bf7b-fa323987001d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:47.467339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0082b37-864b-4c10-bf7b-fa323987001d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:47.467620 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0082b37-864b-4c10-bf7b-fa323987001d None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:47.468426 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0082b37-864b-4c10-bf7b-fa323987001d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:47.468791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0082b37-864b-4c10-bf7b-fa323987001d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:47.470283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0082b37-864b-4c10-bf7b-fa323987001d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:47.479859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373aa19f-3796-4efe-9ed5-59fb491c294c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:47.481327 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373aa19f-3796-4efe-9ed5-59fb491c294c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:47.493096 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 305/597] 10.222.0.78 () {56 vars in 1139 bytes} [Mon Sep 19 11:05:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:47.497305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-373aa19f-3796-4efe-9ed5-59fb491c294c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:47.497305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:47.497305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:47.497305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:47.497305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:47.497305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:47.497305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:47.497305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:47.497305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:47.497305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:47.497305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:47.497305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:47.497305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:47.497305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:47.497305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:47.497305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:47.498250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:47.498250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:47.498250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:47.498250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:47.498250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:47.498250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:47.498250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:47.498250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:47.498250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:47.498250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:47.498250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:47.498250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:47.498250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:47.498250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:47.498250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:47.498250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:47.498250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:47.499129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:47.499129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:47.499129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:47.499129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 293/598] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:47.499484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0082b37-864b-4c10-bf7b-fa323987001d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:47.504297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373aa19f-3796-4efe-9ed5-59fb491c294c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:47.519991 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:47.643657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373aa19f-3796-4efe-9ed5-59fb491c294c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:47.650107 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0082b37-864b-4c10-bf7b-fa323987001d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:47.685925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b78a28-d426-4df0-b69c-a2f064cada5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:47.686736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b78a28-d426-4df0-b69c-a2f064cada5c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:47.687119 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b78a28-d426-4df0-b69c-a2f064cada5c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:47.687524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b78a28-d426-4df0-b69c-a2f064cada5c None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:47.688375 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b78a28-d426-4df0-b69c-a2f064cada5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:47.688786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b78a28-d426-4df0-b69c-a2f064cada5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:47.690323 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b78a28-d426-4df0-b69c-a2f064cada5c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:47.706923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 294/599] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:47.713265 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b78a28-d426-4df0-b69c-a2f064cada5c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:47.713663 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa38e1b-87e9-440c-95a0-ecc69ac72a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:47.714328 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa38e1b-87e9-440c-95a0-ecc69ac72a84 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:47.714610 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa38e1b-87e9-440c-95a0-ecc69ac72a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:47.714882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa38e1b-87e9-440c-95a0-ecc69ac72a84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:47.730871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa38e1b-87e9-440c-95a0-ecc69ac72a84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:47.732299 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa38e1b-87e9-440c-95a0-ecc69ac72a84 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:47.734472 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:47.750571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-6aa38e1b-87e9-440c-95a0-ecc69ac72a84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:47.750571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:47.750571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:47.750571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:47.750571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:47.750571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:47.750571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:47.750571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:47.750571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:47.750571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:47.750571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:47.750571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:47.750571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:47.750571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:47.750571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:47.750571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:47.751584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:47.751584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:47.751584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:47.751584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:47.751584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:47.751584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:47.751584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:47.751584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:47.751584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:47.751584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:47.751584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:47.751584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:47.751584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:47.751584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:47.751584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:47.751584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:47.751584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:47.752709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:47.752709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:47.752709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:47.752709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 306/600] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:47.757891 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa38e1b-87e9-440c-95a0-ecc69ac72a84 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:47.834736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:47.870979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b78a28-d426-4df0-b69c-a2f064cada5c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:47.887620 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa38e1b-87e9-440c-95a0-ecc69ac72a84 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:47.909300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0247a5e-2f07-4c95-a081-073a9b5b30d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:47.910211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0247a5e-2f07-4c95-a081-073a9b5b30d1 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:47.910643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0247a5e-2f07-4c95-a081-073a9b5b30d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:47.911063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0247a5e-2f07-4c95-a081-073a9b5b30d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:47.918796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0247a5e-2f07-4c95-a081-073a9b5b30d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:47.920639 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0247a5e-2f07-4c95-a081-073a9b5b30d1 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:47.927577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcda7d4-cc62-4406-9649-f0a8fc768d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:47.928194 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcda7d4-cc62-4406-9649-f0a8fc768d06 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:47.928473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcda7d4-cc62-4406-9649-f0a8fc768d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:47.928745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcda7d4-cc62-4406-9649-f0a8fc768d06 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:47.929443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcda7d4-cc62-4406-9649-f0a8fc768d06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:47.929751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcda7d4-cc62-4406-9649-f0a8fc768d06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:47.931780 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcda7d4-cc62-4406-9649-f0a8fc768d06 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:47.934860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-e0247a5e-2f07-4c95-a081-073a9b5b30d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:47.934860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:47.934860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:47.934860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:47.934860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:47.934860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:47.934860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:47.934860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:47.934860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:47.934860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:47.934860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:47.934860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:47.934860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:05:47.934860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:05:47.934860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:05:47.935948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:05:47.935948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:47.935948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:47.935948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:47.935948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:47.935948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:47.935948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:47.935948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:47.935948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:47.935948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:47.935948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:47.935948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:47.935948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:47.935948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:47.935948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:47.935948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:47.935948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:47.936978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:05:47.936978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:05:47.936978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:05:47.936978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:05:47.936978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:05:47.936978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:05:47.936978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:47.936978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:47.943411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 295/601] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:47.944280 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 307/602] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:47.945131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcda7d4-cc62-4406-9649-f0a8fc768d06 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:47.945392 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9aea0-9571-4bbd-bf68-e7022cb84ac5 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:47.946381 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9aea0-9571-4bbd-bf68-e7022cb84ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:47.946802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9aea0-9571-4bbd-bf68-e7022cb84ac5 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:47.948504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 296/603] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:47.951030 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9aea0-9571-4bbd-bf68-e7022cb84ac5 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:48.023617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:48.068866 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9aea0-9571-4bbd-bf68-e7022cb84ac5 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:48.080693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcda7d4-cc62-4406-9649-f0a8fc768d06 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:48.114047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:48.125198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b108762-6da1-4098-aacf-f86c55808337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:48.125459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a3b09-7baf-4021-be4d-6d307ec93cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:48.125841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b108762-6da1-4098-aacf-f86c55808337 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:48.126121 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b108762-6da1-4098-aacf-f86c55808337 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:48.126390 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b108762-6da1-4098-aacf-f86c55808337 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:48.126703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a3b09-7baf-4021-be4d-6d307ec93cfe None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:48.127403 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a3b09-7baf-4021-be4d-6d307ec93cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:48.127859 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a3b09-7baf-4021-be4d-6d307ec93cfe None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:48.128729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a3b09-7baf-4021-be4d-6d307ec93cfe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:48.129250 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a3b09-7baf-4021-be4d-6d307ec93cfe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:48.130980 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a3b09-7baf-4021-be4d-6d307ec93cfe None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:48.134368 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b108762-6da1-4098-aacf-f86c55808337 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:48.135867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b108762-6da1-4098-aacf-f86c55808337 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:48.146088 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 308/604] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:05:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:48.147836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096702e9-914f-4a74-8092-5fe4d14c2045 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:48.148304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096702e9-914f-4a74-8092-5fe4d14c2045 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:48.149029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096702e9-914f-4a74-8092-5fe4d14c2045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:48.149340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-0b108762-6da1-4098-aacf-f86c55808337 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:48.149340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:48.149340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:48.149340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:48.149340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:48.149340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:48.149340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:48.149340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:48.149340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:48.149340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:48.149340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:48.149340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:48.149340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:05:48.149340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:05:48.149340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:05:48.150485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:05:48.150485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:48.150485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:48.150485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:48.150485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:48.150485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:48.150485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:48.150485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:48.150485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:48.150485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:48.150485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:48.150485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:48.150485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:48.150485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:48.150485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:48.150485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:48.150485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:48.151432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:05:48.151432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:05:48.151432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:05:48.151432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:05:48.151432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:05:48.151432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:05:48.151432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:48.151432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:48.159250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 297/605] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:48.160402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b108762-6da1-4098-aacf-f86c55808337 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:48.176393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-096702e9-914f-4a74-8092-5fe4d14c2045 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:48.177951 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-096702e9-914f-4a74-8092-5fe4d14c2045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:48.274481 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 309/606] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:48.275459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096702e9-914f-4a74-8092-5fe4d14c2045 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:48.306124 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b108762-6da1-4098-aacf-f86c55808337 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:48.335886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:48.384484 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd350b0-4b46-40c6-acca-667ac2d5b440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:48.385524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd350b0-4b46-40c6-acca-667ac2d5b440 None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:48.386001 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd350b0-4b46-40c6-acca-667ac2d5b440 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:48.386517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd350b0-4b46-40c6-acca-667ac2d5b440 None admin] PATH_INFO: `/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/558e3b6dd9754c57ad028aa431a68b80/roles/28365abffd3d4ff6874bce2e0c561f1d` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:48.396164 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096702e9-914f-4a74-8092-5fe4d14c2045 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:48.424549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd350b0-4b46-40c6-acca-667ac2d5b440 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8563e86b7c94ff29e39fb20a7969551, user_id=558e3b6dd9754c57ad028aa431a68b80, role_id=28365abffd3d4ff6874bce2e0c561f1d)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:48.426670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd350b0-4b46-40c6-acca-667ac2d5b440 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:48.435479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52da2f-dca3-455a-a115-62be3fc055f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:48.436092 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52da2f-dca3-455a-a115-62be3fc055f9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:48.436418 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52da2f-dca3-455a-a115-62be3fc055f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:48.436696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52da2f-dca3-455a-a115-62be3fc055f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:48.437421 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52da2f-dca3-455a-a115-62be3fc055f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:48.437699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52da2f-dca3-455a-a115-62be3fc055f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:48.439129 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52da2f-dca3-455a-a115-62be3fc055f9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:48.455411 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 310/607] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:05:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:48.456976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52da2f-dca3-455a-a115-62be3fc055f9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:48.558964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52da2f-dca3-455a-a115-62be3fc055f9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:48.590154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:48.595101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59a255-7ce3-4ac8-80f2-1d70259fb583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:48.595801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59a255-7ce3-4ac8-80f2-1d70259fb583 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:48.596082 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59a255-7ce3-4ac8-80f2-1d70259fb583 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:48.596357 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59a255-7ce3-4ac8-80f2-1d70259fb583 None admin] PATH_INFO: `/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/dbf0049a3e3a442e945f6207477a7fd0/roles/28365abffd3d4ff6874bce2e0c561f1d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:48.611567 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59a255-7ce3-4ac8-80f2-1d70259fb583 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8563e86b7c94ff29e39fb20a7969551, user_id=dbf0049a3e3a442e945f6207477a7fd0, role_id=28365abffd3d4ff6874bce2e0c561f1d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:48.616611 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59a255-7ce3-4ac8-80f2-1d70259fb583 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:48.663979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 298/608] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:05:48 2022] PUT /identity/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/558e3b6dd9754c57ad028aa431a68b80/roles/28365abffd3d4ff6874bce2e0c561f1d => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:05:48.665372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd350b0-4b46-40c6-acca-667ac2d5b440 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:48.691701 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 311/609] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:05:48 2022] PUT /identity/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/dbf0049a3e3a442e945f6207477a7fd0/roles/28365abffd3d4ff6874bce2e0c561f1d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:05:48.728476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:48.810005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd350b0-4b46-40c6-acca-667ac2d5b440 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:48.852160 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9e296-7d6f-4c06-bcbd-3e261259e72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:48.853144 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9e296-7d6f-4c06-bcbd-3e261259e72e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:48.853589 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9e296-7d6f-4c06-bcbd-3e261259e72e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:48.854078 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9e296-7d6f-4c06-bcbd-3e261259e72e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:48.866796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9e296-7d6f-4c06-bcbd-3e261259e72e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:48.872407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-3ee9e296-7d6f-4c06-bcbd-3e261259e72e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:48.872407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:48.872407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:48.872407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:48.872407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:48.872407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:48.872407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:48.872407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:48.872407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:48.872407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:48.872407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:48.872407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:48.872407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:48.872407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:48.872407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:48.872407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:48.873677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:48.873677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:48.873677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:48.873677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:48.873677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:48.873677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:48.873677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:48.873677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:48.873677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:48.873677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:48.873677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:48.873677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:48.873677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:48.873677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:48.873677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:48.873677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:48.874752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:48.874752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:48.874752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:48.874752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:48.874752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:48.874752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:48.874752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:48.874752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:48.874752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:48.874752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:48.874752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:48.874752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:48.877667 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 299/610] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:48.884283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59a255-7ce3-4ac8-80f2-1d70259fb583 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:49.048413 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59a255-7ce3-4ac8-80f2-1d70259fb583 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:49.085339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9e4ca-21d3-4907-8bf1-25c8bf2aeeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:49.086029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9e4ca-21d3-4907-8bf1-25c8bf2aeeb4 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:49.086483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9e4ca-21d3-4907-8bf1-25c8bf2aeeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:49.087579 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9e4ca-21d3-4907-8bf1-25c8bf2aeeb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:49.088390 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9e4ca-21d3-4907-8bf1-25c8bf2aeeb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:49.088810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9e4ca-21d3-4907-8bf1-25c8bf2aeeb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:49.090269 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9e4ca-21d3-4907-8bf1-25c8bf2aeeb4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:49.099370 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 312/611] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:05:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:49.104848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9e296-7d6f-4c06-bcbd-3e261259e72e None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:49.119963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3b7b8-4f84-4f6e-b081-145fa8594e7f None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:49.120373 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3b7b8-4f84-4f6e-b081-145fa8594e7f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:49.120750 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3b7b8-4f84-4f6e-b081-145fa8594e7f None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:49.122444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 313/612] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:05:49.127784 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995fc45-2537-4ea0-8dee-1eae53e27e7d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:49.128182 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995fc45-2537-4ea0-8dee-1eae53e27e7d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:49.128568 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995fc45-2537-4ea0-8dee-1eae53e27e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:49.147733 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4995fc45-2537-4ea0-8dee-1eae53e27e7d None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:49.150433 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4995fc45-2537-4ea0-8dee-1eae53e27e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:49.162756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:49.253191 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 314/613] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 19 11:05:49.298755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9e296-7d6f-4c06-bcbd-3e261259e72e None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:49.320251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:49.338843 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe85580-2cdb-447e-b43b-c1c4a117be79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:49.346220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe85580-2cdb-447e-b43b-c1c4a117be79 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:49.346516 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe85580-2cdb-447e-b43b-c1c4a117be79 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:49.346795 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe85580-2cdb-447e-b43b-c1c4a117be79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:49.358550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe85580-2cdb-447e-b43b-c1c4a117be79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:49.360031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe85580-2cdb-447e-b43b-c1c4a117be79 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:49.380257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-dbe85580-2cdb-447e-b43b-c1c4a117be79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:49.380257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:49.380257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:49.380257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:49.380257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:49.380257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:49.380257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:49.380257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:49.380257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:49.380257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:49.380257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:49.380257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:49.380257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:49.380257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:49.380257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:49.380257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:49.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:49.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:49.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:49.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:49.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:49.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:49.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:49.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:49.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:49.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:49.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:49.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:49.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:49.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:49.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:49.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:49.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:49.382225 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:49.382225 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:49.382225 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:49.382225 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 300/614] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:49.388400 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995fc45-2537-4ea0-8dee-1eae53e27e7d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:49.547212 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995fc45-2537-4ea0-8dee-1eae53e27e7d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:49.584563 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6531f9-3b72-47d8-8f07-9e920bcc0a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:49.585179 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6531f9-3b72-47d8-8f07-9e920bcc0a21 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:49.585459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6531f9-3b72-47d8-8f07-9e920bcc0a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:49.585731 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6531f9-3b72-47d8-8f07-9e920bcc0a21 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:49.586418 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6531f9-3b72-47d8-8f07-9e920bcc0a21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:49.586721 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6531f9-3b72-47d8-8f07-9e920bcc0a21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:49.588180 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6531f9-3b72-47d8-8f07-9e920bcc0a21 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:49.603361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 315/615] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:49.607317 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe85580-2cdb-447e-b43b-c1c4a117be79 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:49.699931 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6531f9-3b72-47d8-8f07-9e920bcc0a21 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:49.709827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:49.794661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe85580-2cdb-447e-b43b-c1c4a117be79 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:49.832448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c70f2-31db-4a09-87d5-b637cbe7f07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:49.833084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c70f2-31db-4a09-87d5-b637cbe7f07d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:49.833363 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c70f2-31db-4a09-87d5-b637cbe7f07d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:49.833636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c70f2-31db-4a09-87d5-b637cbe7f07d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:49.842343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c70f2-31db-4a09-87d5-b637cbe7f07d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:49.843839 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c70f2-31db-4a09-87d5-b637cbe7f07d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:49.855547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-552c70f2-31db-4a09-87d5-b637cbe7f07d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:05:49.855547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:49.855547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:49.855547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:49.855547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:49.855547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:49.855547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:49.855547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:49.855547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:49.855547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:49.855547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:49.855547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:49.855547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:05:49.855547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:05:49.855547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:05:49.855547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:49.856566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:49.857534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:49.857534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:49.857534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:05:49.857534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:05:49.857534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:05:49.857534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:05:49.857534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:05:49.857534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:05:49.857534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:05:49.857534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:49.861278 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 301/616] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:05:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:49.862557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca8599-1fe0-4675-9b59-9015a84f6fe7 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:49.862843 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca8599-1fe0-4675-9b59-9015a84f6fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:49.863113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca8599-1fe0-4675-9b59-9015a84f6fe7 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:49.864350 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6531f9-3b72-47d8-8f07-9e920bcc0a21 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:49.868898 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 302/617] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:49.868898 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca8599-1fe0-4675-9b59-9015a84f6fe7 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:49.900427 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:49.907524 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ade0e-40fa-4702-b140-50d49b6a70c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:49.908170 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ade0e-40fa-4702-b140-50d49b6a70c3 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:49.908446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ade0e-40fa-4702-b140-50d49b6a70c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:49.908719 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ade0e-40fa-4702-b140-50d49b6a70c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:49.919346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ade0e-40fa-4702-b140-50d49b6a70c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:49.924756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-440ade0e-40fa-4702-b140-50d49b6a70c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:49.924756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:49.924756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:49.924756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:49.924756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:49.924756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:49.924756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:49.924756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:49.924756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:49.924756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:49.924756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:49.924756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:49.924756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:49.924756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:49.924756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:49.924756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:49.925710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:49.925710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:49.925710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:49.925710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:49.925710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:49.925710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:49.925710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:49.925710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:49.925710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:49.925710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:49.925710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:49.925710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:49.925710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:49.925710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:49.925710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:49.925710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:49.926553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:49.926553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:49.926553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:49.926553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:49.926553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:49.926553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:49.926553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:49.926553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:49.926553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:49.926553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:49.926553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:49.926553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:49.926553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 316/618] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:49.927948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebb624-bb1e-4d38-8f23-908acd73d150 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:49.927948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebb624-bb1e-4d38-8f23-908acd73d150 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:49.928116 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebb624-bb1e-4d38-8f23-908acd73d150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:49.949001 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-22ebb624-bb1e-4d38-8f23-908acd73d150 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:49.950291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-22ebb624-bb1e-4d38-8f23-908acd73d150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:50.016028 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:50.020852 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca8599-1fe0-4675-9b59-9015a84f6fe7 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:50.024389 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 317/619] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:50.028007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebb624-bb1e-4d38-8f23-908acd73d150 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:50.066376 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7dadb-8198-4749-b6f2-1dde50170b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:50.067259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7dadb-8198-4749-b6f2-1dde50170b67 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:50.067706 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7dadb-8198-4749-b6f2-1dde50170b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:50.068128 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7dadb-8198-4749-b6f2-1dde50170b67 None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:50.069002 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7dadb-8198-4749-b6f2-1dde50170b67 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:50.069466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7dadb-8198-4749-b6f2-1dde50170b67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:50.072285 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7dadb-8198-4749-b6f2-1dde50170b67 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:50.096852 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 303/620] 10.222.0.78 () {56 vars in 1135 bytes} [Mon Sep 19 11:05:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:05:50.102216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7dadb-8198-4749-b6f2-1dde50170b67 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:50.142118 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebb624-bb1e-4d38-8f23-908acd73d150 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:50.183015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:50.200616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401af030-524c-4d63-a255-19ad34e42477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:50.201256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401af030-524c-4d63-a255-19ad34e42477 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:50.201540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401af030-524c-4d63-a255-19ad34e42477 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:50.201811 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401af030-524c-4d63-a255-19ad34e42477 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:50.202491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401af030-524c-4d63-a255-19ad34e42477 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:50.202785 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401af030-524c-4d63-a255-19ad34e42477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:50.204236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401af030-524c-4d63-a255-19ad34e42477 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:50.222307 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 318/621] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:05:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:50.225596 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401af030-524c-4d63-a255-19ad34e42477 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:50.228922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7dadb-8198-4749-b6f2-1dde50170b67 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:50.292970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def66611-e622-45e2-80a4-bdc2a0adb0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:50.293783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def66611-e622-45e2-80a4-bdc2a0adb0fd None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:50.294194 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def66611-e622-45e2-80a4-bdc2a0adb0fd None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:50.294558 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def66611-e622-45e2-80a4-bdc2a0adb0fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:50.306453 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def66611-e622-45e2-80a4-bdc2a0adb0fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:50.308117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def66611-e622-45e2-80a4-bdc2a0adb0fd None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:50.324359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-def66611-e622-45e2-80a4-bdc2a0adb0fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:50.324359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:50.324359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:50.324359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:50.324359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:50.324359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:50.324359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:50.324359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:50.324359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:50.324359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:50.324359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:50.324359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:50.324359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:50.324359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:50.324359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:50.324359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:50.326410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:50.326410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:50.326410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:50.326410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:50.326410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:50.326410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:50.326410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:50.326410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:50.326410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:50.326410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:50.326410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:50.326410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:50.326410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:50.326410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:50.326410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:50.326410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:50.326410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:50.327440 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:50.327440 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:50.327440 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:50.327629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 304/622] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:50.333484 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def66611-e622-45e2-80a4-bdc2a0adb0fd None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:50.343092 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:50.413002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401af030-524c-4d63-a255-19ad34e42477 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:50.454187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf9789-6956-42ea-9c82-3a020c926014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:50.455458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf9789-6956-42ea-9c82-3a020c926014 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:50.455781 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf9789-6956-42ea-9c82-3a020c926014 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:50.456065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf9789-6956-42ea-9c82-3a020c926014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:50.467685 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def66611-e622-45e2-80a4-bdc2a0adb0fd None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:50.471978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf9789-6956-42ea-9c82-3a020c926014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:50.473443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf9789-6956-42ea-9c82-3a020c926014 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:50.491826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-8cbf9789-6956-42ea-9c82-3a020c926014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:50.491826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:50.491826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:50.491826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:50.491826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:50.491826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:50.491826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:50.491826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:50.491826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:50.491826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:50.491826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:50.491826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:50.491826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:50.491826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:50.491826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:50.491826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:50.493424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:50.493424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:50.493424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:50.493424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:50.493424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:50.493424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:50.493424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:50.493424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:50.493424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:50.493424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:50.493424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:50.493424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:50.493424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:50.493424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:50.493424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:50.493424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:50.493424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:50.494308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:50.494308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:50.494308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:50.494308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 319/623] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:50.495058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf9789-6956-42ea-9c82-3a020c926014 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:50.497169 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:50.519087 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c1726-95ba-4737-bb90-b71bfdc9b6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:50.520497 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c1726-95ba-4737-bb90-b71bfdc9b6b8 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:50.525680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c1726-95ba-4737-bb90-b71bfdc9b6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:50.525680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c1726-95ba-4737-bb90-b71bfdc9b6b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:50.525680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c1726-95ba-4737-bb90-b71bfdc9b6b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:50.525680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c1726-95ba-4737-bb90-b71bfdc9b6b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:50.525680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c1726-95ba-4737-bb90-b71bfdc9b6b8 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:50.546453 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 305/624] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:50.546453 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c1726-95ba-4737-bb90-b71bfdc9b6b8 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:50.615812 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf9789-6956-42ea-9c82-3a020c926014 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:50.629208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:50.656229 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc106a00-4b09-4d0a-9edd-f4818204058a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:50.657648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc106a00-4b09-4d0a-9edd-f4818204058a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:50.658382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc106a00-4b09-4d0a-9edd-f4818204058a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:50.658760 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc106a00-4b09-4d0a-9edd-f4818204058a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:50.671525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc106a00-4b09-4d0a-9edd-f4818204058a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:50.677776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-fc106a00-4b09-4d0a-9edd-f4818204058a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:50.677776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:50.677776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:50.677776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:50.677776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:50.677776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:50.677776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:50.677776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:50.677776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:50.677776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:50.677776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:50.677776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:50.677776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:50.677776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:50.677776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:50.677776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:50.678934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:50.678934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:50.678934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:50.678934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:50.678934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:50.678934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:50.678934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:50.678934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:50.678934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:50.678934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:50.678934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:50.678934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:50.678934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:50.678934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:50.678934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:50.678934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:50.680052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:50.680052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:50.680052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:50.680052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:50.680052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:50.680052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:50.680052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:50.680052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:50.680052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:50.680052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:50.680052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:50.680052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:50.686154 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c1726-95ba-4737-bb90-b71bfdc9b6b8 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:50.687021 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 320/625] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:50.688230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc106a00-4b09-4d0a-9edd-f4818204058a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:50.733278 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3418f6f-eeed-49f3-9bcb-36e766ca74da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:50.734149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3418f6f-eeed-49f3-9bcb-36e766ca74da None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:50.734571 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3418f6f-eeed-49f3-9bcb-36e766ca74da None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:50.735010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3418f6f-eeed-49f3-9bcb-36e766ca74da None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:50.735822 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3418f6f-eeed-49f3-9bcb-36e766ca74da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:50.736263 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3418f6f-eeed-49f3-9bcb-36e766ca74da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:50.737788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3418f6f-eeed-49f3-9bcb-36e766ca74da None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:50.754370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 306/626] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:50.755484 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3418f6f-eeed-49f3-9bcb-36e766ca74da None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:50.803991 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:50.820375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc106a00-4b09-4d0a-9edd-f4818204058a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:50.862784 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e249f8d5-ec42-4aa9-adea-99e91e749ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:50.863819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249f8d5-ec42-4aa9-adea-99e91e749ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:50.864266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249f8d5-ec42-4aa9-adea-99e91e749ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:50.864680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249f8d5-ec42-4aa9-adea-99e91e749ce7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:50.869940 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3418f6f-eeed-49f3-9bcb-36e766ca74da None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:50.873278 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e249f8d5-ec42-4aa9-adea-99e91e749ce7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:50.874917 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e249f8d5-ec42-4aa9-adea-99e91e749ce7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:50.889196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-e249f8d5-ec42-4aa9-adea-99e91e749ce7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:50.889196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:50.889196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:50.889196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:50.889196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:50.889196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:50.889196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:50.889196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:50.889196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:50.889196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:50.889196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:50.889196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:50.889196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:05:50.889196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:05:50.889196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:05:50.890300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:05:50.890300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:50.890300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:50.890300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:50.890300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:50.890300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:50.890300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:50.890300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:50.890300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:50.890300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:50.890300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:50.890300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:50.890300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:50.890300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:50.890300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:50.890300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:50.890300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:50.891394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:05:50.891394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:05:50.891394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:05:50.891394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:05:50.891394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:05:50.891394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:05:50.891394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:50.891394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:50.898268 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 321/627] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:50.900386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e249f8d5-ec42-4aa9-adea-99e91e749ce7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:50.910773 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d84065-5010-418a-bd5a-9ef7ef34cb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:50.911622 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d84065-5010-418a-bd5a-9ef7ef34cb9a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:50.912020 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d84065-5010-418a-bd5a-9ef7ef34cb9a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:50.912452 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d84065-5010-418a-bd5a-9ef7ef34cb9a None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:50.913234 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d84065-5010-418a-bd5a-9ef7ef34cb9a None admin] RBAC: Adding query filter params (name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:50.913657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d84065-5010-418a-bd5a-9ef7ef34cb9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:50.915796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d84065-5010-418a-bd5a-9ef7ef34cb9a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:50.929592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 307/628] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:05:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:50.931383 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d84065-5010-418a-bd5a-9ef7ef34cb9a None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:50.940133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:51.027933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e249f8d5-ec42-4aa9-adea-99e91e749ce7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:51.040022 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d84065-5010-418a-bd5a-9ef7ef34cb9a None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:51.041985 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:51.073667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e468b79-cb25-4c2f-9e8c-453e7a8b1beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:51.074599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e468b79-cb25-4c2f-9e8c-453e7a8b1beb None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:51.075033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e468b79-cb25-4c2f-9e8c-453e7a8b1beb None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:51.075751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e468b79-cb25-4c2f-9e8c-453e7a8b1beb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:51.082194 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a2712-f9f9-47de-81d7-c7cf3c9ca47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:51.082959 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a2712-f9f9-47de-81d7-c7cf3c9ca47d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:51.083394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a2712-f9f9-47de-81d7-c7cf3c9ca47d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:51.083777 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a2712-f9f9-47de-81d7-c7cf3c9ca47d None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:51.084731 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a2712-f9f9-47de-81d7-c7cf3c9ca47d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:51.085182 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a2712-f9f9-47de-81d7-c7cf3c9ca47d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:51.086017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e468b79-cb25-4c2f-9e8c-453e7a8b1beb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:51.087028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a2712-f9f9-47de-81d7-c7cf3c9ca47d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:51.088276 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e468b79-cb25-4c2f-9e8c-453e7a8b1beb None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:51.100232 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 308/629] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:05:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:51.101243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a2712-f9f9-47de-81d7-c7cf3c9ca47d None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:51.103049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-7e468b79-cb25-4c2f-9e8c-453e7a8b1beb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:05:51.103049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:51.103049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:51.103049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:51.103049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:51.103049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:51.103049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:51.103049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:51.103049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:51.103049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:51.103049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:51.103049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:51.103049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:05:51.103049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:05:51.103049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:05:51.103049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:51.104207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:51.105534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:51.105534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:51.105534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:05:51.105534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:05:51.105534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:05:51.105534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:05:51.105534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:05:51.105534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:05:51.105534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:05:51.105534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:51.107745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 322/630] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:51.113767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e468b79-cb25-4c2f-9e8c-453e7a8b1beb None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:51.237387 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:51.239524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a2712-f9f9-47de-81d7-c7cf3c9ca47d None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:51.249075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e468b79-cb25-4c2f-9e8c-453e7a8b1beb None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:51.279431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b6ccd-82b3-4b07-a12b-5f21ba2da75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:51.280297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b6ccd-82b3-4b07-a12b-5f21ba2da75a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:51.280486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b6ccd-82b3-4b07-a12b-5f21ba2da75a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:51.280753 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b6ccd-82b3-4b07-a12b-5f21ba2da75a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:51.292810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141f91f-4c5a-4fc8-afbb-d16a82846b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:51.293425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b6ccd-82b3-4b07-a12b-5f21ba2da75a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:51.294026 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141f91f-4c5a-4fc8-afbb-d16a82846b69 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:51.294516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141f91f-4c5a-4fc8-afbb-d16a82846b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:51.294898 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b6ccd-82b3-4b07-a12b-5f21ba2da75a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:51.295235 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141f91f-4c5a-4fc8-afbb-d16a82846b69 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:51.296317 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141f91f-4c5a-4fc8-afbb-d16a82846b69 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:51.296805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141f91f-4c5a-4fc8-afbb-d16a82846b69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:51.299211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141f91f-4c5a-4fc8-afbb-d16a82846b69 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:51.312235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-886b6ccd-82b3-4b07-a12b-5f21ba2da75a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:51.312235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:51.312235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:51.312235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:51.312235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:51.312235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:51.312235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:51.312235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:51.312235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:51.312235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:51.312235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:51.312235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:51.312235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:51.312235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:51.312235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:51.312235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:51.313245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:51.313245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:51.313245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:51.313245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:51.313245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:51.313245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:51.313245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:51.313245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:51.313245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:51.313245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:51.313245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:51.313245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:51.313245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:51.313245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:51.313245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:51.313245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:51.313245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:51.314136 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:51.314136 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:51.314136 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:51.318420 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 309/631] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:51.319575 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b6ccd-82b3-4b07-a12b-5f21ba2da75a None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:51.325127 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 323/632] 10.222.0.78 () {56 vars in 1137 bytes} [Mon Sep 19 11:05:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:51.326448 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141f91f-4c5a-4fc8-afbb-d16a82846b69 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:51.340447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:51.438815 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b6ccd-82b3-4b07-a12b-5f21ba2da75a None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:51.442498 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141f91f-4c5a-4fc8-afbb-d16a82846b69 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:51.473983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda3ac7-2a19-4fac-81ca-c9077e095481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:51.474647 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda3ac7-2a19-4fac-81ca-c9077e095481 None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:51.474924 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda3ac7-2a19-4fac-81ca-c9077e095481 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:51.475218 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda3ac7-2a19-4fac-81ca-c9077e095481 None admin] PATH_INFO: `/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/cf462b2aa3fc4e2ca58851549cf0c644/roles/28365abffd3d4ff6874bce2e0c561f1d` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:51.478186 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa8729-12ad-4bf6-93d6-8f9b07db342a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:51.478800 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa8729-12ad-4bf6-93d6-8f9b07db342a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:51.479079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa8729-12ad-4bf6-93d6-8f9b07db342a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:51.479384 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa8729-12ad-4bf6-93d6-8f9b07db342a None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:51.480031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa8729-12ad-4bf6-93d6-8f9b07db342a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:51.480333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa8729-12ad-4bf6-93d6-8f9b07db342a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:51.481731 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa8729-12ad-4bf6-93d6-8f9b07db342a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:51.492498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda3ac7-2a19-4fac-81ca-c9077e095481 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8563e86b7c94ff29e39fb20a7969551, user_id=cf462b2aa3fc4e2ca58851549cf0c644, role_id=28365abffd3d4ff6874bce2e0c561f1d)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:51.493871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda3ac7-2a19-4fac-81ca-c9077e095481 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:51.493980 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 324/633] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:51.494803 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa8729-12ad-4bf6-93d6-8f9b07db342a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:51.506324 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:51.542434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 310/634] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:05:51 2022] PUT /identity/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/cf462b2aa3fc4e2ca58851549cf0c644/roles/28365abffd3d4ff6874bce2e0c561f1d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:05:51.543796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4eee1a-ff1f-4fcd-8b4a-61bb238dd713 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:51.544090 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4eee1a-ff1f-4fcd-8b4a-61bb238dd713 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:51.544363 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4eee1a-ff1f-4fcd-8b4a-61bb238dd713 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:51.545618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 311/635] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:51.546792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b1bce-5801-4b47-996b-d550d7296229 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:51.547128 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b1bce-5801-4b47-996b-d550d7296229 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:51.547429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b1bce-5801-4b47-996b-d550d7296229 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:51.548880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 312/636] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:51.549523 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b1bce-5801-4b47-996b-d550d7296229 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:51.623104 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa8729-12ad-4bf6-93d6-8f9b07db342a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:51.624173 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:51.672272 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bfec19-9e4e-4c18-b719-184e365b930e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:51.673289 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfec19-9e4e-4c18-b719-184e365b930e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:51.673731 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfec19-9e4e-4c18-b719-184e365b930e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:51.674370 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfec19-9e4e-4c18-b719-184e365b930e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:51.675828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b1bce-5801-4b47-996b-d550d7296229 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:51.718673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35adf3-49d3-4dcf-a72e-775f59da0e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:51.719357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35adf3-49d3-4dcf-a72e-775f59da0e61 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:51.719639 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35adf3-49d3-4dcf-a72e-775f59da0e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:51.719916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35adf3-49d3-4dcf-a72e-775f59da0e61 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:51.733615 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35adf3-49d3-4dcf-a72e-775f59da0e61 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:51.735914 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35adf3-49d3-4dcf-a72e-775f59da0e61 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:51.752909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-1b35adf3-49d3-4dcf-a72e-775f59da0e61 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:05:51.752909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:51.752909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:51.752909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:51.752909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:51.752909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:51.752909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:51.752909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:51.752909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:51.752909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:51.752909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:51.752909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:51.752909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:05:51.752909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:05:51.752909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:05:51.752909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:51.753944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:51.754929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:51.754929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:51.754929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:05:51.754929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:05:51.754929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:05:51.754929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:05:51.754929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:05:51.754929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:05:51.754929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:05:51.754929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:51.754929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 313/637] 10.222.0.78 () {56 vars in 1093 bytes} [Mon Sep 19 11:05:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:51.755684 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50859d04-cd5f-4ffe-9f31-06c4ee6f5719 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:51.755877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50859d04-cd5f-4ffe-9f31-06c4ee6f5719 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:51.756146 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50859d04-cd5f-4ffe-9f31-06c4ee6f5719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:51.772830 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bfec19-9e4e-4c18-b719-184e365b930e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:51.774846 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bfec19-9e4e-4c18-b719-184e365b930e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:51.776108 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-50859d04-cd5f-4ffe-9f31-06c4ee6f5719 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:51.777310 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-50859d04-cd5f-4ffe-9f31-06c4ee6f5719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:51.785692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:51.852612 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 314/638] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:51.854572 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa744b0f-822a-4be7-baed-32b0916c44bc None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:51.855104 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa744b0f-822a-4be7-baed-32b0916c44bc None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:51.855626 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa744b0f-822a-4be7-baed-32b0916c44bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:51.875031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-aa744b0f-822a-4be7-baed-32b0916c44bc None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:51.877061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-aa744b0f-822a-4be7-baed-32b0916c44bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:51.889720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 325/639] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:51.890680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bfec19-9e4e-4c18-b719-184e365b930e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:51.941457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:51.963861 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 315/640] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:51.965620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa744b0f-822a-4be7-baed-32b0916c44bc None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:52.015374 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bfec19-9e4e-4c18-b719-184e365b930e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:52.027503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:52.064304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448015d-6ee2-4729-98d1-65fdd16ffb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:52.065459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448015d-6ee2-4729-98d1-65fdd16ffb2c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:52.065753 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448015d-6ee2-4729-98d1-65fdd16ffb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:52.066039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448015d-6ee2-4729-98d1-65fdd16ffb2c None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:52.066787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448015d-6ee2-4729-98d1-65fdd16ffb2c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:52.067102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448015d-6ee2-4729-98d1-65fdd16ffb2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:52.068914 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448015d-6ee2-4729-98d1-65fdd16ffb2c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:52.101279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa744b0f-822a-4be7-baed-32b0916c44bc None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:52.102885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 326/641] 10.222.0.78 () {56 vars in 1133 bytes} [Mon Sep 19 11:05:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:52.109345 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448015d-6ee2-4729-98d1-65fdd16ffb2c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:52.164844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5dc22-3e90-4e12-9b74-cb86755b5f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:52.165740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5dc22-3e90-4e12-9b74-cb86755b5f05 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:52.166019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5dc22-3e90-4e12-9b74-cb86755b5f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:52.166286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5dc22-3e90-4e12-9b74-cb86755b5f05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:52.181533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b5dc22-3e90-4e12-9b74-cb86755b5f05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:52.184627 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b5dc22-3e90-4e12-9b74-cb86755b5f05 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:52.215544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-75b5dc22-3e90-4e12-9b74-cb86755b5f05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:52.215544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:52.215544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:52.215544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:52.215544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:52.215544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:52.215544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:52.215544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:52.215544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:52.215544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:52.215544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:52.215544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:52.215544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:52.215544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:52.215544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:52.215544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:52.216551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:52.216551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:52.216551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:52.216551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:52.216551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:52.216551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:52.216551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:52.216551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:52.216551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:52.216551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:52.216551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:52.216551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:52.216551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:52.216551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:52.216551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:52.216551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:52.216551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:52.217423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:52.217423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:52.217423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:52.222068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 316/642] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:52.223014 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5dc22-3e90-4e12-9b74-cb86755b5f05 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:52.242928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448015d-6ee2-4729-98d1-65fdd16ffb2c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:52.271885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:52.282882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada95ea-9937-47e8-a5db-2a431df4d1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:52.283794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada95ea-9937-47e8-a5db-2a431df4d1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:52.284217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada95ea-9937-47e8-a5db-2a431df4d1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:52.284629 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada95ea-9937-47e8-a5db-2a431df4d1c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:52.328128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:52.358417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5dc22-3e90-4e12-9b74-cb86755b5f05 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:52.373490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada95ea-9937-47e8-a5db-2a431df4d1c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:52.374953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada95ea-9937-47e8-a5db-2a431df4d1c7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:52.405303 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0f1b3-17b6-4f9b-93f0-06fa9fdd8852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:52.406065 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0f1b3-17b6-4f9b-93f0-06fa9fdd8852 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:52.406393 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0f1b3-17b6-4f9b-93f0-06fa9fdd8852 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:52.406672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0f1b3-17b6-4f9b-93f0-06fa9fdd8852 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:52.417832 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0f1b3-17b6-4f9b-93f0-06fa9fdd8852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:52.422850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-38e0f1b3-17b6-4f9b-93f0-06fa9fdd8852 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:52.422850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:52.422850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:52.422850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:52.422850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:52.422850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:52.422850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:52.422850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:52.422850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:52.422850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:52.422850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:52.422850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:52.422850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:52.422850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:52.422850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:52.422850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:52.423976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:52.423976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:52.423976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:52.423976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:52.423976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:52.423976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:52.423976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:52.423976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:52.423976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:52.423976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:52.423976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:52.423976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:52.423976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:52.423976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:52.423976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:52.423976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:52.424956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:52.424956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:52.424956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:52.424956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:52.424956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:52.424956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:52.424956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:52.424956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:52.424956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:52.424956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:52.424956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:52.424956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:52.424956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 317/643] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:52.425660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0f1b3-17b6-4f9b-93f0-06fa9fdd8852 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:52.498024 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 327/644] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:52.499576 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada95ea-9937-47e8-a5db-2a431df4d1c7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:52.541228 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:52.549175 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0f1b3-17b6-4f9b-93f0-06fa9fdd8852 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:52.580397 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:52.587189 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd26cb-e825-4a32-9f87-bed67c2e53f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:52.587803 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd26cb-e825-4a32-9f87-bed67c2e53f1 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:52.588086 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd26cb-e825-4a32-9f87-bed67c2e53f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:52.588364 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd26cb-e825-4a32-9f87-bed67c2e53f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:52.589048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd26cb-e825-4a32-9f87-bed67c2e53f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:52.589314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd26cb-e825-4a32-9f87-bed67c2e53f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:52.590688 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd26cb-e825-4a32-9f87-bed67c2e53f1 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:52.602882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 318/645] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:52.604243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd26cb-e825-4a32-9f87-bed67c2e53f1 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:52.630657 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada95ea-9937-47e8-a5db-2a431df4d1c7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:52.668672 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee820d-d292-4e91-b37e-f097fee7273f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:52.669310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee820d-d292-4e91-b37e-f097fee7273f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:52.669592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee820d-d292-4e91-b37e-f097fee7273f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:52.669862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee820d-d292-4e91-b37e-f097fee7273f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:52.680989 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ee820d-d292-4e91-b37e-f097fee7273f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:52.686444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-a0ee820d-d292-4e91-b37e-f097fee7273f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:52.686444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:52.686444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:52.686444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:52.686444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:52.686444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:52.686444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:52.686444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:52.686444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:52.686444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:52.686444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:52.686444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:52.686444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:52.686444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:52.686444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:52.686444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:52.691876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:52.691876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:52.691876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:52.691876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:52.691876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:52.691876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:52.691876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:52.691876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:52.691876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:52.691876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:52.691876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:52.691876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:52.691876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:52.691876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:52.691876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:52.691876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:52.693077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:52.693077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:52.693077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:52.693077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:52.693077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:52.693077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:52.693077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:52.693077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:52.693077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:52.693077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:52.693077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:52.693077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:52.693077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 328/646] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:52.693077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee820d-d292-4e91-b37e-f097fee7273f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:52.720741 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd26cb-e825-4a32-9f87-bed67c2e53f1 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:52.759381 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50fa2ca-edf8-42ab-9424-c6cbcb845358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:52.760004 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fa2ca-edf8-42ab-9424-c6cbcb845358 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:52.760287 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fa2ca-edf8-42ab-9424-c6cbcb845358 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:52.760562 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fa2ca-edf8-42ab-9424-c6cbcb845358 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:52.761228 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50fa2ca-edf8-42ab-9424-c6cbcb845358 None admin] RBAC: Adding query filter params (name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:52.761540 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50fa2ca-edf8-42ab-9424-c6cbcb845358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:52.762909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50fa2ca-edf8-42ab-9424-c6cbcb845358 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:52.776252 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 319/647] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:05:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:52.777151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50fa2ca-edf8-42ab-9424-c6cbcb845358 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:52.797486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:52.798111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee820d-d292-4e91-b37e-f097fee7273f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:52.826246 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:52.836974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe65a72-88a9-41da-853d-549ba5557ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:52.837642 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe65a72-88a9-41da-853d-549ba5557ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:52.838014 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe65a72-88a9-41da-853d-549ba5557ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:52.838293 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe65a72-88a9-41da-853d-549ba5557ea8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:52.849173 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe65a72-88a9-41da-853d-549ba5557ea8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:52.850807 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe65a72-88a9-41da-853d-549ba5557ea8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:52.867434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-afe65a72-88a9-41da-853d-549ba5557ea8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:52.867434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:52.867434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:52.867434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:52.867434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:52.867434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:52.867434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:52.867434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:52.867434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:52.867434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:52.867434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:52.867434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:52.867434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:52.867434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:52.867434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:52.867434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:52.869133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:52.869133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:52.869133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:52.869133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:52.869133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:52.869133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:52.869133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:52.869133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:52.869133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:52.869133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:52.869133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:52.869133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:52.869133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:52.869133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:52.869133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:52.869133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:52.869133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:52.870252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:52.870252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:52.870252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:52.876630 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 329/648] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:52.876630 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe65a72-88a9-41da-853d-549ba5557ea8 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:52.902973 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50fa2ca-edf8-42ab-9424-c6cbcb845358 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:52.941971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea55ad-7d19-4645-bf70-0904ec561819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:52.942610 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea55ad-7d19-4645-bf70-0904ec561819 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:52.942891 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea55ad-7d19-4645-bf70-0904ec561819 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:52.943184 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea55ad-7d19-4645-bf70-0904ec561819 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:52.950404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea55ad-7d19-4645-bf70-0904ec561819 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:52.951765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea55ad-7d19-4645-bf70-0904ec561819 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:52.964240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-20ea55ad-7d19-4645-bf70-0904ec561819 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:52.964240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:52.964240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:52.964240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:52.964240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:52.964240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:52.964240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:52.964240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:52.964240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:52.964240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:52.964240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:52.964240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:52.964240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:05:52.964240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:05:52.964240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:05:52.965286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:05:52.965286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:52.965286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:52.965286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:52.965286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:52.965286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:52.965286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:52.965286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:52.965286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:52.965286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:52.965286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:52.965286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:52.965286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:52.965286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:52.965286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:52.965286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:52.965286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:52.966162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:05:52.966162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:05:52.966162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:05:52.966162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:05:52.966162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:05:52.966162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:05:52.966162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:52.966162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:52.966162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 320/649] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:52.966162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea55ad-7d19-4645-bf70-0904ec561819 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:52.988269 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe65a72-88a9-41da-853d-549ba5557ea8 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:53.026683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2fc24-15dd-42cb-ac15-efb7ed4e737c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:53.027343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2fc24-15dd-42cb-ac15-efb7ed4e737c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:53.027630 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2fc24-15dd-42cb-ac15-efb7ed4e737c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:53.027905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2fc24-15dd-42cb-ac15-efb7ed4e737c None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:53.028584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2fc24-15dd-42cb-ac15-efb7ed4e737c None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:53.028893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2fc24-15dd-42cb-ac15-efb7ed4e737c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:53.030301 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2fc24-15dd-42cb-ac15-efb7ed4e737c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:53.039047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 330/650] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:05:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:53.040588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2fc24-15dd-42cb-ac15-efb7ed4e737c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:53.057386 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:53.079636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea55ad-7d19-4645-bf70-0904ec561819 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:53.090797 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:53.120381 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76776a-4d23-4600-a55b-2aaf2b360509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:53.121057 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76776a-4d23-4600-a55b-2aaf2b360509 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:53.121343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76776a-4d23-4600-a55b-2aaf2b360509 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:53.121625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76776a-4d23-4600-a55b-2aaf2b360509 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:53.133181 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76776a-4d23-4600-a55b-2aaf2b360509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:53.134662 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76776a-4d23-4600-a55b-2aaf2b360509 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:53.151720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-da76776a-4d23-4600-a55b-2aaf2b360509 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:53.151720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:53.151720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:53.151720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:53.151720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:53.151720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:53.151720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:53.151720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:53.151720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:53.151720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:53.151720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:53.151720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:53.151720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:53.151720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:53.151720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:53.151720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:53.152882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:53.152882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:53.152882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:53.152882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:53.152882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:53.152882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:53.152882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:53.152882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:53.152882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:53.152882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:53.152882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:53.152882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:53.152882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:53.152882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:53.152882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:53.152882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:53.152882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:53.153807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:53.153807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:53.153807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:53.153807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 321/651] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:53.153807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76776a-4d23-4600-a55b-2aaf2b360509 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:53.165311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2fc24-15dd-42cb-ac15-efb7ed4e737c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:53.200449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e2c28-2563-4e80-ab2a-40a3c406d330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:53.201211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e2c28-2563-4e80-ab2a-40a3c406d330 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:53.201493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e2c28-2563-4e80-ab2a-40a3c406d330 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:53.201793 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e2c28-2563-4e80-ab2a-40a3c406d330 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:53.202443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e2c28-2563-4e80-ab2a-40a3c406d330 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:53.202745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e2c28-2563-4e80-ab2a-40a3c406d330 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:53.204312 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e2c28-2563-4e80-ab2a-40a3c406d330 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:53.217760 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 331/652] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:53.218598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e2c28-2563-4e80-ab2a-40a3c406d330 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:53.268121 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76776a-4d23-4600-a55b-2aaf2b360509 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:53.302069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:53.308794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ba511-b2d6-4a58-949e-f1e57f6f7645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:53.309350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ba511-b2d6-4a58-949e-f1e57f6f7645 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:53.309636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ba511-b2d6-4a58-949e-f1e57f6f7645 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:53.309906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ba511-b2d6-4a58-949e-f1e57f6f7645 None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:53.310600 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ba511-b2d6-4a58-949e-f1e57f6f7645 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:53.311224 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ba511-b2d6-4a58-949e-f1e57f6f7645 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:53.312388 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ba511-b2d6-4a58-949e-f1e57f6f7645 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:53.322310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:53.325226 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 322/653] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:05:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:53.325875 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ba511-b2d6-4a58-949e-f1e57f6f7645 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:53.331139 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e2c28-2563-4e80-ab2a-40a3c406d330 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:53.373907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46feba76-2dd4-41d2-9714-a6a39c2d723e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:53.374585 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46feba76-2dd4-41d2-9714-a6a39c2d723e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:53.374840 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46feba76-2dd4-41d2-9714-a6a39c2d723e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:53.375123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46feba76-2dd4-41d2-9714-a6a39c2d723e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:53.386592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46feba76-2dd4-41d2-9714-a6a39c2d723e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:53.388151 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46feba76-2dd4-41d2-9714-a6a39c2d723e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:53.408695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-46feba76-2dd4-41d2-9714-a6a39c2d723e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:53.408695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:53.408695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:53.408695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:53.408695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:53.408695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:53.408695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:53.408695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:53.408695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:53.408695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:53.408695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:53.408695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:53.408695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:53.408695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:53.408695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:53.408695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:53.409788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:53.409788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:53.409788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:53.409788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:53.409788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:53.409788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:53.409788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:53.409788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:53.409788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:53.409788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:53.409788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:53.409788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:53.409788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:53.409788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:53.409788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:53.409788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:53.409788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:53.410686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:53.410686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:53.410686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:53.410686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 332/654] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:53.410686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46feba76-2dd4-41d2-9714-a6a39c2d723e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:53.441217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ba511-b2d6-4a58-949e-f1e57f6f7645 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:53.481493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4b68f-35fb-4516-a6f0-5e210a742ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:53.482411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4b68f-35fb-4516-a6f0-5e210a742ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:53.482911 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4b68f-35fb-4516-a6f0-5e210a742ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:53.483411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4b68f-35fb-4516-a6f0-5e210a742ee6 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:53.484304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4b68f-35fb-4516-a6f0-5e210a742ee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:53.484789 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4b68f-35fb-4516-a6f0-5e210a742ee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:53.486354 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4b68f-35fb-4516-a6f0-5e210a742ee6 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:53.498791 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 323/655] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:53.500037 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4b68f-35fb-4516-a6f0-5e210a742ee6 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:53.526567 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46feba76-2dd4-41d2-9714-a6a39c2d723e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:53.555183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:53.561697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42c5c0-197c-405f-9b57-429a1d4eb11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:53.562286 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42c5c0-197c-405f-9b57-429a1d4eb11d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:53.562568 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42c5c0-197c-405f-9b57-429a1d4eb11d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:53.562845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42c5c0-197c-405f-9b57-429a1d4eb11d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:53.570013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42c5c0-197c-405f-9b57-429a1d4eb11d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:53.572061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42c5c0-197c-405f-9b57-429a1d4eb11d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:53.583899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-2b42c5c0-197c-405f-9b57-429a1d4eb11d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:53.583899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:53.583899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:53.583899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:53.583899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:53.583899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:53.583899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:53.583899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:53.583899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:53.583899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:53.583899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:53.583899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:53.583899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:05:53.583899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:05:53.583899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:05:53.584844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:05:53.584844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:53.584844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:53.584844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:53.584844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:53.584844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:53.584844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:53.584844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:53.584844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:53.584844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:53.584844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:53.584844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:53.584844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:53.584844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:53.584844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:53.584844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:53.584844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:53.585751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:05:53.585751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:05:53.585751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:05:53.585751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:05:53.585751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:05:53.585751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:05:53.585751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:53.585751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:53.585751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 333/656] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:53.587256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42c5c0-197c-405f-9b57-429a1d4eb11d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:53.607570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:53.617709 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4b68f-35fb-4516-a6f0-5e210a742ee6 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:53.665500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14358714-645d-45e3-a1cb-a7b0cc145802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:53.666116 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14358714-645d-45e3-a1cb-a7b0cc145802 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:53.666397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14358714-645d-45e3-a1cb-a7b0cc145802 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:53.666906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14358714-645d-45e3-a1cb-a7b0cc145802 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:53.667418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14358714-645d-45e3-a1cb-a7b0cc145802 None admin] RBAC: Adding query filter params (role.id=28365abffd3d4ff6874bce2e0c561f1d, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=1ab94f6e08644fa7a76b9057e46d122d) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:53.667724 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14358714-645d-45e3-a1cb-a7b0cc145802 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:53.669156 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14358714-645d-45e3-a1cb-a7b0cc145802 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:53.683198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 324/657] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:05:53 2022] GET /identity/v3/role_assignments?user.id=1ab94f6e08644fa7a76b9057e46d122d&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=28365abffd3d4ff6874bce2e0c561f1d => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:53.683198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14358714-645d-45e3-a1cb-a7b0cc145802 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:53.740061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42c5c0-197c-405f-9b57-429a1d4eb11d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:53.790587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d202801-ec19-41c7-ac6a-377a3de9078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:53.791467 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d202801-ec19-41c7-ac6a-377a3de9078f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:53.791819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d202801-ec19-41c7-ac6a-377a3de9078f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:53.792388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d202801-ec19-41c7-ac6a-377a3de9078f None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:53.793250 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d202801-ec19-41c7-ac6a-377a3de9078f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:53.793631 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d202801-ec19-41c7-ac6a-377a3de9078f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:53.795130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d202801-ec19-41c7-ac6a-377a3de9078f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:53.811928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 334/658] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:53.812995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d202801-ec19-41c7-ac6a-377a3de9078f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:53.814250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14358714-645d-45e3-a1cb-a7b0cc145802 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:53.833940 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:53.857282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2271488-a926-4cb4-bfb2-53f491cb173d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:53.857680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2271488-a926-4cb4-bfb2-53f491cb173d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:53.857955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2271488-a926-4cb4-bfb2-53f491cb173d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:53.858319 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2271488-a926-4cb4-bfb2-53f491cb173d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:53.868302 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2271488-a926-4cb4-bfb2-53f491cb173d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:53.870313 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2271488-a926-4cb4-bfb2-53f491cb173d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:53.882695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-f2271488-a926-4cb4-bfb2-53f491cb173d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:05:53.882695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:53.882695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:53.882695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:53.882695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:53.882695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:53.882695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:53.882695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:53.882695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:53.882695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:53.882695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:53.882695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:53.882695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:05:53.882695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:05:53.882695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:05:53.882695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:53.884841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:53.885949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:53.885949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:53.885949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:05:53.885949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:05:53.885949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:05:53.885949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:05:53.885949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:05:53.885949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:05:53.885949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:05:53.885949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:53.885949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 325/659] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:53.885949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2271488-a926-4cb4-bfb2-53f491cb173d None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:53.907236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:53.943136 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d202801-ec19-41c7-ac6a-377a3de9078f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:53.981665 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c1243-f9f5-444b-a6da-66e68f2e8f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:53.982270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c1243-f9f5-444b-a6da-66e68f2e8f7c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:53.982548 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c1243-f9f5-444b-a6da-66e68f2e8f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:53.982819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c1243-f9f5-444b-a6da-66e68f2e8f7c None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:53.983615 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c1243-f9f5-444b-a6da-66e68f2e8f7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:53.983836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c1243-f9f5-444b-a6da-66e68f2e8f7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:53.985263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c1243-f9f5-444b-a6da-66e68f2e8f7c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:53.998965 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 335/660] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:05:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:54.000238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bacc35-9233-40a8-a6f4-e25e2eac4907 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:54.000530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bacc35-9233-40a8-a6f4-e25e2eac4907 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:54.000809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bacc35-9233-40a8-a6f4-e25e2eac4907 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:54.006232 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 336/661] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:54.006232 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bacc35-9233-40a8-a6f4-e25e2eac4907 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:54.013950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2271488-a926-4cb4-bfb2-53f491cb173d None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:54.051072 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58021aa-a2f3-44c4-ba45-961cfcb71990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:54.051740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58021aa-a2f3-44c4-ba45-961cfcb71990 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:54.052020 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58021aa-a2f3-44c4-ba45-961cfcb71990 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:54.052401 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58021aa-a2f3-44c4-ba45-961cfcb71990 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:54.061297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58021aa-a2f3-44c4-ba45-961cfcb71990 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:54.062963 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58021aa-a2f3-44c4-ba45-961cfcb71990 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:54.076220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-f58021aa-a2f3-44c4-ba45-961cfcb71990 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:05:54.076220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:54.076220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:54.076220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:54.076220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:54.076220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:54.076220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:54.076220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:54.076220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:54.076220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:54.076220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:54.076220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:54.076220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:05:54.076220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:05:54.076220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:05:54.076220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:54.077404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:54.078418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:54.078418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:54.078418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:05:54.078418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:05:54.078418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:05:54.078418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:05:54.078418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:05:54.078418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:05:54.078418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:05:54.078418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:54.078418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 326/662] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:54.078418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58021aa-a2f3-44c4-ba45-961cfcb71990 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:54.120349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:54.125710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bacc35-9233-40a8-a6f4-e25e2eac4907 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:54.156256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:54.162510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af752b-691f-4a14-ac8f-443bce1342ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:54.163137 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af752b-691f-4a14-ac8f-443bce1342ea None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:54.163464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af752b-691f-4a14-ac8f-443bce1342ea None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:54.163743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af752b-691f-4a14-ac8f-443bce1342ea None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:54.165027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af752b-691f-4a14-ac8f-443bce1342ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:54.165339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af752b-691f-4a14-ac8f-443bce1342ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:54.166795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af752b-691f-4a14-ac8f-443bce1342ea None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:54.187208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 337/663] 10.222.0.78 () {56 vars in 1137 bytes} [Mon Sep 19 11:05:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:54.188503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c944e3a-8b29-43d7-bdec-f3f2081e581f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:54.188818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c944e3a-8b29-43d7-bdec-f3f2081e581f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:54.189116 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c944e3a-8b29-43d7-bdec-f3f2081e581f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:54.198305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58021aa-a2f3-44c4-ba45-961cfcb71990 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:54.207607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8c944e3a-8b29-43d7-bdec-f3f2081e581f None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:54.208803 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8c944e3a-8b29-43d7-bdec-f3f2081e581f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:54.235653 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c829e9-1610-41b7-bfd3-b2bb1af6b3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:54.236263 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c829e9-1610-41b7-bfd3-b2bb1af6b3c5 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:54.236547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c829e9-1610-41b7-bfd3-b2bb1af6b3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:54.236822 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c829e9-1610-41b7-bfd3-b2bb1af6b3c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:54.237555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c829e9-1610-41b7-bfd3-b2bb1af6b3c5 None admin] RBAC: Adding query filter params (role.id=28365abffd3d4ff6874bce2e0c561f1d, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=aba1968529574fe5b4c5d114c874f428) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:54.237865 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c829e9-1610-41b7-bfd3-b2bb1af6b3c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:54.239345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c829e9-1610-41b7-bfd3-b2bb1af6b3c5 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:54.255210 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 327/664] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:05:54 2022] GET /identity/v3/role_assignments?user.id=aba1968529574fe5b4c5d114c874f428&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=28365abffd3d4ff6874bce2e0c561f1d => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:54.255210 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c829e9-1610-41b7-bfd3-b2bb1af6b3c5 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:54.296466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 338/665] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:54.300140 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c944e3a-8b29-43d7-bdec-f3f2081e581f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:54.439015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:54.443458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c829e9-1610-41b7-bfd3-b2bb1af6b3c5 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:54.476434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c944e3a-8b29-43d7-bdec-f3f2081e581f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:54.500594 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82588888-9346-4f55-af35-7a224882edc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:54.501990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82588888-9346-4f55-af35-7a224882edc1 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:54.502278 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82588888-9346-4f55-af35-7a224882edc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:54.502556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82588888-9346-4f55-af35-7a224882edc1 None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:54.503349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82588888-9346-4f55-af35-7a224882edc1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:54.503660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82588888-9346-4f55-af35-7a224882edc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:54.505103 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82588888-9346-4f55-af35-7a224882edc1 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:54.506651 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:54.528996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592d09e-30e8-4c82-b2d0-405fa9a6034f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:54.530330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592d09e-30e8-4c82-b2d0-405fa9a6034f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:54.530915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592d09e-30e8-4c82-b2d0-405fa9a6034f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:54.531493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592d09e-30e8-4c82-b2d0-405fa9a6034f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:54.531962 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 328/666] 10.222.0.78 () {56 vars in 1139 bytes} [Mon Sep 19 11:05:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:54.538456 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82588888-9346-4f55-af35-7a224882edc1 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:54.629682 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592d09e-30e8-4c82-b2d0-405fa9a6034f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:54.631385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592d09e-30e8-4c82-b2d0-405fa9a6034f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:54.678894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82588888-9346-4f55-af35-7a224882edc1 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:54.716561 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadd18b-85c6-4258-84ed-ca63ec179cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:54.717095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadd18b-85c6-4258-84ed-ca63ec179cce None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:54.717377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadd18b-85c6-4258-84ed-ca63ec179cce None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:54.717697 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadd18b-85c6-4258-84ed-ca63ec179cce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:54.745517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:54.745732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 339/667] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:54.746597 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592d09e-30e8-4c82-b2d0-405fa9a6034f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:54.768905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:54.805645 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eadd18b-85c6-4258-84ed-ca63ec179cce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:54.807486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eadd18b-85c6-4258-84ed-ca63ec179cce None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:54.856967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592d09e-30e8-4c82-b2d0-405fa9a6034f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:54.895210 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85e37c-af6e-4583-81b8-015e45550a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:54.895882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85e37c-af6e-4583-81b8-015e45550a1e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:54.896164 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85e37c-af6e-4583-81b8-015e45550a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:54.896439 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85e37c-af6e-4583-81b8-015e45550a1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:54.911909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85e37c-af6e-4583-81b8-015e45550a1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:54.918586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-4c85e37c-af6e-4583-81b8-015e45550a1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:54.918586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:54.918586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:54.918586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:54.918586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:54.918586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:54.918586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:54.918586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:54.918586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:54.918586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:54.918586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:54.918586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:54.918586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:54.918586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:54.918586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:54.918586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:54.919514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:54.919514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:54.919514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:54.919514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:54.919514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:54.919514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:54.919514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:54.919514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:54.919514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:54.919514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:54.919514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:54.919514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:54.919514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:54.919514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:54.919514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:54.919514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:54.920346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:54.920346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:54.920346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:54.920346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:54.920346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:54.920346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:54.920346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:54.920346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:54.920346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:54.920346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:54.920346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:05:54.920346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:54.920346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 340/668] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:05:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:54.921484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85e37c-af6e-4583-81b8-015e45550a1e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:54.939644 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 329/669] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:54.940507 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadd18b-85c6-4258-84ed-ca63ec179cce None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:54.999760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:55.027732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:55.050442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85e37c-af6e-4583-81b8-015e45550a1e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:55.057238 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadd18b-85c6-4258-84ed-ca63ec179cce None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:55.092300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbe80d-e4ee-488c-847d-a62777c628ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:55.093056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbe80d-e4ee-488c-847d-a62777c628ad None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:55.093344 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbe80d-e4ee-488c-847d-a62777c628ad None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:55.093646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbe80d-e4ee-488c-847d-a62777c628ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:55.096245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85eb4d7-47e3-4fb7-9a8c-49142f106f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:55.097144 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85eb4d7-47e3-4fb7-9a8c-49142f106f74 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:55.097569 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85eb4d7-47e3-4fb7-9a8c-49142f106f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:55.097986 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85eb4d7-47e3-4fb7-9a8c-49142f106f74 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:55.098791 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85eb4d7-47e3-4fb7-9a8c-49142f106f74 None admin] RBAC: Adding query filter params (name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:55.099264 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85eb4d7-47e3-4fb7-9a8c-49142f106f74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:55.100887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85eb4d7-47e3-4fb7-9a8c-49142f106f74 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:55.104778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbe80d-e4ee-488c-847d-a62777c628ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:55.106234 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbe80d-e4ee-488c-847d-a62777c628ad None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:55.110593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 330/670] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:05:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:55.111772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85eb4d7-47e3-4fb7-9a8c-49142f106f74 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:55.124462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-8fbbe80d-e4ee-488c-847d-a62777c628ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:55.124462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:55.124462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:55.124462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:55.124462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:55.124462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:55.124462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:55.124462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:55.124462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:55.124462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:55.124462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:55.124462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:55.124462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:55.124462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:55.124462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:55.124462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:55.125584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:55.125584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:55.125584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:55.125584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:55.125584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:55.125584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:55.125584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:55.125584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:55.125584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:55.125584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:55.125584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:55.125584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:55.125584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:55.125584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:55.125584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:55.125584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:55.125584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:55.126924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:55.126924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:55.126924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:55.126924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 341/671] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:55.126924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbe80d-e4ee-488c-847d-a62777c628ad None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:55.227102 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85eb4d7-47e3-4fb7-9a8c-49142f106f74 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:55.246476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:55.248639 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbe80d-e4ee-488c-847d-a62777c628ad None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:55.266070 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e5d6f-37d9-4d7d-9b78-8eceb2e1103d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:55.267086 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e5d6f-37d9-4d7d-9b78-8eceb2e1103d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:55.267599 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e5d6f-37d9-4d7d-9b78-8eceb2e1103d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:55.268057 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e5d6f-37d9-4d7d-9b78-8eceb2e1103d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:55.280606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e5d6f-37d9-4d7d-9b78-8eceb2e1103d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:55.282401 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e5d6f-37d9-4d7d-9b78-8eceb2e1103d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:55.284119 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:55.290079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9721e8-bb48-45b6-bfcd-05b8cd94fa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:55.290733 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9721e8-bb48-45b6-bfcd-05b8cd94fa88 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:55.291032 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9721e8-bb48-45b6-bfcd-05b8cd94fa88 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:55.291900 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9721e8-bb48-45b6-bfcd-05b8cd94fa88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:55.299725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-ea2e5d6f-37d9-4d7d-9b78-8eceb2e1103d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:55.299725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:55.299725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:55.299725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:55.299725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:55.299725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:55.299725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:55.299725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:55.299725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:55.299725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:55.299725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:55.299725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:55.299725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:55.299725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:55.299725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:55.299725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:55.300763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:55.300763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:55.300763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:55.300763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:55.300763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:55.300763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:55.300763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:55.300763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:55.300763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:55.300763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:55.300763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:55.300763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:55.300763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:55.300763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:55.300763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:55.300763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:55.300763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:55.301704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:55.301704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:55.301704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:55.308313 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9721e8-bb48-45b6-bfcd-05b8cd94fa88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:55.308313 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9721e8-bb48-45b6-bfcd-05b8cd94fa88 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:55.308625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 331/672] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:55.309626 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e5d6f-37d9-4d7d-9b78-8eceb2e1103d None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:55.322850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-0f9721e8-bb48-45b6-bfcd-05b8cd94fa88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:55.322850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:55.322850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:55.322850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:55.322850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:55.322850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:55.322850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:55.322850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:55.322850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:55.322850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:55.322850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:55.322850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:55.322850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:55.322850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:55.322850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:55.322850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:55.324142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:55.324142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:55.324142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:55.324142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:55.324142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:55.324142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:55.324142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:55.324142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:55.324142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:55.324142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:55.324142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:55.324142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:55.324142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:55.324142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:55.324142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:55.324142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:55.324142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:55.325141 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:55.325141 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:55.325141 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:55.325141 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 342/673] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:55.325141 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9721e8-bb48-45b6-bfcd-05b8cd94fa88 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:55.426094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e5d6f-37d9-4d7d-9b78-8eceb2e1103d None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:55.448693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9721e8-bb48-45b6-bfcd-05b8cd94fa88 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:55.472931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77803c1-9b99-4723-aea9-f54d446f5ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:55.473554 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77803c1-9b99-4723-aea9-f54d446f5ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:55.473841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77803c1-9b99-4723-aea9-f54d446f5ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:55.474118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77803c1-9b99-4723-aea9-f54d446f5ba5 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:55.475031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77803c1-9b99-4723-aea9-f54d446f5ba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:55.475377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77803c1-9b99-4723-aea9-f54d446f5ba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:55.477285 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77803c1-9b99-4723-aea9-f54d446f5ba5 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:55.488839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fb520-f1dd-4252-9fb3-9ced8f5ca1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:55.489815 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fb520-f1dd-4252-9fb3-9ced8f5ca1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:55.490279 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fb520-f1dd-4252-9fb3-9ced8f5ca1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:55.490732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fb520-f1dd-4252-9fb3-9ced8f5ca1b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:55.491584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fb520-f1dd-4252-9fb3-9ced8f5ca1b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:55.492125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fb520-f1dd-4252-9fb3-9ced8f5ca1b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:55.493351 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 332/674] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:55.494295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77803c1-9b99-4723-aea9-f54d446f5ba5 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:55.494825 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fb520-f1dd-4252-9fb3-9ced8f5ca1b2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:55.516392 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 343/675] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:55.518018 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fb520-f1dd-4252-9fb3-9ced8f5ca1b2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:55.535448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:55.575148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:55.625300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77803c1-9b99-4723-aea9-f54d446f5ba5 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:55.638572 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fb520-f1dd-4252-9fb3-9ced8f5ca1b2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:55.670070 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968961d5-cff5-46b9-8b15-6590825616d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:55.671086 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968961d5-cff5-46b9-8b15-6590825616d5 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:55.672137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968961d5-cff5-46b9-8b15-6590825616d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:55.672137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968961d5-cff5-46b9-8b15-6590825616d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:55.672620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968961d5-cff5-46b9-8b15-6590825616d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:55.673351 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968961d5-cff5-46b9-8b15-6590825616d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:55.675559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968961d5-cff5-46b9-8b15-6590825616d5 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:55.684862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e9b06-509a-4094-9749-16ca5d9b8a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:55.685818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e9b06-509a-4094-9749-16ca5d9b8a43 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:55.686358 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e9b06-509a-4094-9749-16ca5d9b8a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:55.686809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e9b06-509a-4094-9749-16ca5d9b8a43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:55.691451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 333/676] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:55.692321 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968961d5-cff5-46b9-8b15-6590825616d5 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:55.695270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e9b06-509a-4094-9749-16ca5d9b8a43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:55.697922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e9b06-509a-4094-9749-16ca5d9b8a43 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:55.713809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-0a3e9b06-509a-4094-9749-16ca5d9b8a43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:55.713809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:55.713809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:55.713809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:55.713809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:55.713809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:55.713809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:55.713809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:55.713809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:55.713809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:55.713809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:55.713809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:55.713809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:05:55.713809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:05:55.713809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:05:55.715835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:05:55.715835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:55.715835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:55.715835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:55.715835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:55.715835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:55.715835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:55.715835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:55.715835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:55.715835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:55.715835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:55.715835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:55.715835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:55.715835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:55.715835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:55.715835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:55.715835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:55.716953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:05:55.716953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:05:55.716953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:05:55.716953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:05:55.716953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:05:55.716953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:05:55.716953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:55.716953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:55.724454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 344/677] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:55.724454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e9b06-509a-4094-9749-16ca5d9b8a43 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:55.807676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:55.820172 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968961d5-cff5-46b9-8b15-6590825616d5 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:55.851256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e9b06-509a-4094-9749-16ca5d9b8a43 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:55.854920 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:55.875761 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab140a66-15f4-4cca-bcb7-b12568a4f61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:55.876415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140a66-15f4-4cca-bcb7-b12568a4f61e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:55.876960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140a66-15f4-4cca-bcb7-b12568a4f61e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:55.877236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140a66-15f4-4cca-bcb7-b12568a4f61e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:55.885487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab140a66-15f4-4cca-bcb7-b12568a4f61e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:55.886990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab140a66-15f4-4cca-bcb7-b12568a4f61e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:55.896901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef9197-159e-47f7-b9e6-d71cacf60f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:55.897552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef9197-159e-47f7-b9e6-d71cacf60f5a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:55.897839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef9197-159e-47f7-b9e6-d71cacf60f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:55.898116 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef9197-159e-47f7-b9e6-d71cacf60f5a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:55.902977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-ab140a66-15f4-4cca-bcb7-b12568a4f61e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:55.902977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:55.902977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:55.902977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:55.902977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:55.902977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:55.902977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:55.902977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:55.902977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:55.902977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:55.902977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:55.902977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:55.902977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:05:55.902977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:05:55.902977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:05:55.903981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:05:55.903981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:55.903981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:55.903981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:55.903981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:55.903981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:55.903981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:55.903981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:55.903981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:55.903981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:55.903981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:55.903981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:55.903981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:55.903981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:55.903981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:55.903981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:55.903981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:55.904883 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:05:55.904883 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:05:55.904883 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:05:55.904883 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:05:55.904883 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:05:55.904883 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:05:55.904883 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:55.904883 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:55.904883 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 334/678] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:55.905364 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab140a66-15f4-4cca-bcb7-b12568a4f61e None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:55.911939 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef9197-159e-47f7-b9e6-d71cacf60f5a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:55.913477 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef9197-159e-47f7-b9e6-d71cacf60f5a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:55.926591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-21ef9197-159e-47f7-b9e6-d71cacf60f5a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:05:55.926591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:55.926591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:55.926591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:55.926591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:55.926591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:55.926591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:55.926591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:55.926591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:55.926591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:55.926591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:55.926591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:55.926591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:05:55.926591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:05:55.926591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:05:55.926591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:55.928777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:55.931650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:55.931650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:55.931650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:05:55.931650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:05:55.931650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:05:55.931650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:05:55.931650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:05:55.931650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:05:55.931650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:05:55.931650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:55.931650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 345/679] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:05:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:55.931650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef9197-159e-47f7-b9e6-d71cacf60f5a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:56.034373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab140a66-15f4-4cca-bcb7-b12568a4f61e None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:56.035642 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef9197-159e-47f7-b9e6-d71cacf60f5a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:56.071216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc9bfe-4235-4d84-8945-b9b7a529664b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:56.071882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc9bfe-4235-4d84-8945-b9b7a529664b None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:56.072160 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc9bfe-4235-4d84-8945-b9b7a529664b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:56.072430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc9bfe-4235-4d84-8945-b9b7a529664b None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:56.073110 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc9bfe-4235-4d84-8945-b9b7a529664b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:56.073410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc9bfe-4235-4d84-8945-b9b7a529664b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:56.074831 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc9bfe-4235-4d84-8945-b9b7a529664b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:56.076024 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1bff83-eaf4-471e-80cc-e0f619eadbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:56.076825 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bff83-eaf4-471e-80cc-e0f619eadbc8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:56.077468 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bff83-eaf4-471e-80cc-e0f619eadbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:56.077817 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bff83-eaf4-471e-80cc-e0f619eadbc8 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:56.078514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1bff83-eaf4-471e-80cc-e0f619eadbc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:56.078819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1bff83-eaf4-471e-80cc-e0f619eadbc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:56.080302 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1bff83-eaf4-471e-80cc-e0f619eadbc8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:56.092168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 335/680] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:05:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:56.092168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc9bfe-4235-4d84-8945-b9b7a529664b None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:56.097776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 346/681] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:05:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:56.099375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f4965-b4fa-425b-bc0e-67134104a4d4 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:56.100013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f4965-b4fa-425b-bc0e-67134104a4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:56.100351 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f4965-b4fa-425b-bc0e-67134104a4d4 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:56.106303 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 347/682] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:56.106303 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f4965-b4fa-425b-bc0e-67134104a4d4 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:56.111862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:56.134236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:56.204838 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc9bfe-4235-4d84-8945-b9b7a529664b None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:56.221379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f4965-b4fa-425b-bc0e-67134104a4d4 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:56.240801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c64538-8b71-4513-a504-bbc299dfe998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:56.241415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c64538-8b71-4513-a504-bbc299dfe998 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:56.241691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c64538-8b71-4513-a504-bbc299dfe998 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:56.241962 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c64538-8b71-4513-a504-bbc299dfe998 None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:56.242755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c64538-8b71-4513-a504-bbc299dfe998 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:56.243057 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c64538-8b71-4513-a504-bbc299dfe998 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:56.244531 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c64538-8b71-4513-a504-bbc299dfe998 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:56.262448 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27be2bb-d989-4c69-8c09-dadc6bc5f64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:56.267754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27be2bb-d989-4c69-8c09-dadc6bc5f64b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:56.267907 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 336/683] 10.222.0.78 () {56 vars in 1135 bytes} [Mon Sep 19 11:05:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:56.268002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27be2bb-d989-4c69-8c09-dadc6bc5f64b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:56.268002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27be2bb-d989-4c69-8c09-dadc6bc5f64b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:56.268177 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c64538-8b71-4513-a504-bbc299dfe998 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:56.268272 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27be2bb-d989-4c69-8c09-dadc6bc5f64b None admin] RBAC: Adding query filter params (role.id=28365abffd3d4ff6874bce2e0c561f1d, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=558e3b6dd9754c57ad028aa431a68b80) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:56.268272 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27be2bb-d989-4c69-8c09-dadc6bc5f64b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:56.268272 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27be2bb-d989-4c69-8c09-dadc6bc5f64b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:56.282133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 348/684] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:05:56 2022] GET /identity/v3/role_assignments?user.id=558e3b6dd9754c57ad028aa431a68b80&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=28365abffd3d4ff6874bce2e0c561f1d => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:56.282133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955df42-69d7-40d3-a0c1-36fd08a6b89a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:56.282133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955df42-69d7-40d3-a0c1-36fd08a6b89a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:56.282133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955df42-69d7-40d3-a0c1-36fd08a6b89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:56.296799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8955df42-69d7-40d3-a0c1-36fd08a6b89a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:56.298118 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8955df42-69d7-40d3-a0c1-36fd08a6b89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:56.352505 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:56.380262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 349/685] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:56.381318 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955df42-69d7-40d3-a0c1-36fd08a6b89a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:56.404582 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:56.412623 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c64538-8b71-4513-a504-bbc299dfe998 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:56.489471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ea38f-098b-4e96-82fd-6283475ba596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:56.490094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ea38f-098b-4e96-82fd-6283475ba596 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:56.490366 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ea38f-098b-4e96-82fd-6283475ba596 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:56.490634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ea38f-098b-4e96-82fd-6283475ba596 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:56.491395 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ea38f-098b-4e96-82fd-6283475ba596 None admin] RBAC: Adding query filter params (role.id=28365abffd3d4ff6874bce2e0c561f1d, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=dbf0049a3e3a442e945f6207477a7fd0) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:56.491700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ea38f-098b-4e96-82fd-6283475ba596 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:56.493198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ea38f-098b-4e96-82fd-6283475ba596 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:56.506324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 337/686] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:05:56 2022] GET /identity/v3/role_assignments?user.id=dbf0049a3e3a442e945f6207477a7fd0&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=28365abffd3d4ff6874bce2e0c561f1d => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:56.523138 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955df42-69d7-40d3-a0c1-36fd08a6b89a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:56.565209 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcf35f-b9fb-4d14-8892-8e66c889f451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:56.565836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcf35f-b9fb-4d14-8892-8e66c889f451 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:56.566154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcf35f-b9fb-4d14-8892-8e66c889f451 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:56.566380 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcf35f-b9fb-4d14-8892-8e66c889f451 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:56.647823 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcf35f-b9fb-4d14-8892-8e66c889f451 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:56.649793 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcf35f-b9fb-4d14-8892-8e66c889f451 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:56.671901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:56.746179 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ea38f-098b-4e96-82fd-6283475ba596 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:56.776097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 350/687] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:56.783435 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcf35f-b9fb-4d14-8892-8e66c889f451 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:56.876458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ea38f-098b-4e96-82fd-6283475ba596 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:56.911700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:56.912589 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcf35f-b9fb-4d14-8892-8e66c889f451 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:56.923198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc5ed2-9694-4d9e-9993-baaf92dcce99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:56.924569 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc5ed2-9694-4d9e-9993-baaf92dcce99 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:56.925593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc5ed2-9694-4d9e-9993-baaf92dcce99 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:56.926108 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc5ed2-9694-4d9e-9993-baaf92dcce99 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:56.927183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc5ed2-9694-4d9e-9993-baaf92dcce99 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:56.930356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc5ed2-9694-4d9e-9993-baaf92dcce99 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:56.932033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:56.949625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-c1dc5ed2-9694-4d9e-9993-baaf92dcce99 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:05:56.949625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:56.949625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:05:56.949625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:05:56.949625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:05:56.949625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:05:56.949625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:05:56.949625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:05:56.949625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:05:56.949625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:56.949625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:05:56.949625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:56.949625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:56.949625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:56.949625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:56.949625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:56.949625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:56.950713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:56.950713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:56.950713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:56.950713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:56.950713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:56.950713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:56.950713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:05:56.950713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:05:56.950713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:05:56.950713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:05:56.950713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:56.950713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:56.950713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:56.950713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:56.950713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:56.950713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:56.951603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:56.951603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:56.951603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:56.951603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:56.951603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:56.951603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:56.951603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:56.951603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:56.951603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:56.951603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:56.951603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:05:56.951603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:05:56.951603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:05:56.951603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:56.953014 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db6b11-cb2a-4804-a8e8-484f295357dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:56.953014 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db6b11-cb2a-4804-a8e8-484f295357dc None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:56.953014 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db6b11-cb2a-4804-a8e8-484f295357dc None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:56.953014 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db6b11-cb2a-4804-a8e8-484f295357dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:56.955540 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 338/688] 10.222.0.78 () {56 vars in 1129 bytes} [Mon Sep 19 11:05:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:56.957023 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98255dd-b55e-40aa-b5db-5169d8e6395f None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:56.957588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98255dd-b55e-40aa-b5db-5169d8e6395f None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:56.958120 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98255dd-b55e-40aa-b5db-5169d8e6395f None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:56.960480 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 339/689] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:56.967475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98255dd-b55e-40aa-b5db-5169d8e6395f None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:56.967948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db6b11-cb2a-4804-a8e8-484f295357dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:56.969665 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db6b11-cb2a-4804-a8e8-484f295357dc None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:56.986379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-d9db6b11-cb2a-4804-a8e8-484f295357dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:56.986379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:56.986379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:56.986379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:56.986379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:56.986379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:56.986379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:56.986379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:56.986379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:56.986379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:56.986379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:56.986379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:56.986379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:56.986379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:56.986379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:56.986379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:56.987401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:56.987401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:56.987401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:56.987401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:56.987401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:56.987401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:56.987401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:56.987401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:56.987401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:56.987401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:56.987401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:56.987401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:56.987401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:56.987401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:56.987401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:56.987401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:56.987401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:56.988305 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:56.988305 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:56.988305 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:56.988305 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 351/690] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:56.989200 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c5a76-c707-4fdf-a9b5-b55c3102e119 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:56.989484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c5a76-c707-4fdf-a9b5-b55c3102e119 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:56.989760 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c5a76-c707-4fdf-a9b5-b55c3102e119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:57.009812 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-410c5a76-c707-4fdf-a9b5-b55c3102e119 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:57.010978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-410c5a76-c707-4fdf-a9b5-b55c3102e119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:57.087195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 352/691] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:57.087195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c5a76-c707-4fdf-a9b5-b55c3102e119 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:57.106551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98255dd-b55e-40aa-b5db-5169d8e6395f None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:57.180673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01ef21-ebcc-4192-8893-177917eb48ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:57.181332 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01ef21-ebcc-4192-8893-177917eb48ce None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:57.181620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01ef21-ebcc-4192-8893-177917eb48ce None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:57.181897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01ef21-ebcc-4192-8893-177917eb48ce None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:57.182543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01ef21-ebcc-4192-8893-177917eb48ce None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:57.182847 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01ef21-ebcc-4192-8893-177917eb48ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:57.184469 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:57.185255 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01ef21-ebcc-4192-8893-177917eb48ce None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:57.187518 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c5a76-c707-4fdf-a9b5-b55c3102e119 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:57.198381 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 340/692] 10.222.0.78 () {56 vars in 1124 bytes} [Mon Sep 19 11:05:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:57.211592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01ef21-ebcc-4192-8893-177917eb48ce None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:57.229776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8883c3-5b54-4c24-bd5a-3ad8b499c377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:57.230382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8883c3-5b54-4c24-bd5a-3ad8b499c377 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:57.230665 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8883c3-5b54-4c24-bd5a-3ad8b499c377 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:57.230939 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8883c3-5b54-4c24-bd5a-3ad8b499c377 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:57.231629 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8883c3-5b54-4c24-bd5a-3ad8b499c377 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:57.231933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8883c3-5b54-4c24-bd5a-3ad8b499c377 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:57.233775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8883c3-5b54-4c24-bd5a-3ad8b499c377 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:57.247236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 353/693] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:57.252856 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8883c3-5b54-4c24-bd5a-3ad8b499c377 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:57.344232 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:57.362359 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8883c3-5b54-4c24-bd5a-3ad8b499c377 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:57.399613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85a316-e623-4f56-a2bb-c771751a7e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:57.400266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85a316-e623-4f56-a2bb-c771751a7e5a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:57.400545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85a316-e623-4f56-a2bb-c771751a7e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:57.400821 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85a316-e623-4f56-a2bb-c771751a7e5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:57.412167 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85a316-e623-4f56-a2bb-c771751a7e5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:57.414333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85a316-e623-4f56-a2bb-c771751a7e5a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:57.425321 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01ef21-ebcc-4192-8893-177917eb48ce None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:57.431327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-bc85a316-e623-4f56-a2bb-c771751a7e5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:57.431327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:57.431327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:57.431327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:57.431327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:57.431327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:57.431327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:57.431327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:57.431327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:57.431327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:57.431327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:57.431327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:57.431327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:05:57.431327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:05:57.431327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:05:57.432221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:05:57.432221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:57.432221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:57.432221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:57.432221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:57.432221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:57.432221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:57.432221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:57.432221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:57.432221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:57.432221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:57.432221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:57.432221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:57.432221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:57.432221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:57.432221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:57.432221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:57.433157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:05:57.433157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:05:57.433157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:05:57.433157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:05:57.433157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:05:57.433157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:05:57.433157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:57.433157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:57.433157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 354/694] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:57.433661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85a316-e623-4f56-a2bb-c771751a7e5a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:57.459050 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:57.464725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad42e7f-f33e-4cab-bdb5-f8a609d2e6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:57.468377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad42e7f-f33e-4cab-bdb5-f8a609d2e6ee None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:57.468377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad42e7f-f33e-4cab-bdb5-f8a609d2e6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:57.468377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad42e7f-f33e-4cab-bdb5-f8a609d2e6ee None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:57.468377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad42e7f-f33e-4cab-bdb5-f8a609d2e6ee None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:57.468377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad42e7f-f33e-4cab-bdb5-f8a609d2e6ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:57.468876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad42e7f-f33e-4cab-bdb5-f8a609d2e6ee None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:57.475122 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 341/695] 10.222.0.78 () {56 vars in 1124 bytes} [Mon Sep 19 11:05:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:57.476950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad42e7f-f33e-4cab-bdb5-f8a609d2e6ee None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:57.621524 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85a316-e623-4f56-a2bb-c771751a7e5a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:57.692087 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a26c6f-645b-486e-beb9-a4c14ea1b0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:57.693049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a26c6f-645b-486e-beb9-a4c14ea1b0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:57.693515 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a26c6f-645b-486e-beb9-a4c14ea1b0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:57.693969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a26c6f-645b-486e-beb9-a4c14ea1b0a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:57.713523 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad42e7f-f33e-4cab-bdb5-f8a609d2e6ee None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:57.717789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a26c6f-645b-486e-beb9-a4c14ea1b0a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:05:57.725571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-a7a26c6f-645b-486e-beb9-a4c14ea1b0a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:05:57.725571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:57.725571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:57.725571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:57.725571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:57.725571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:57.725571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:57.725571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:57.725571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:57.725571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:57.725571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:57.725571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:57.725571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:05:57.725571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:05:57.725571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:05:57.725571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:05:57.727089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:05:57.727089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:05:57.727089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:05:57.727089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:05:57.727089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:57.727089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:57.727089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:57.727089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:57.727089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:57.727089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:57.727089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:57.727089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:57.727089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:57.727089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:57.727089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:57.727089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:57.728019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:57.728019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:57.728019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:57.728019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:57.728019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:05:57.728019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:05:57.728019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:05:57.728019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:05:57.728019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:05:57.728019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:05:57.728019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:05:57.728019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:57.731084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 355/696] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:05:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:57.738391 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a26c6f-645b-486e-beb9-a4c14ea1b0a6 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:57.771393 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:57.792459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b92b05-04b6-4883-80e7-5c6bd7f0a6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:57.793358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b92b05-04b6-4883-80e7-5c6bd7f0a6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:57.793786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b92b05-04b6-4883-80e7-5c6bd7f0a6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:57.794203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b92b05-04b6-4883-80e7-5c6bd7f0a6d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:57.795051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b92b05-04b6-4883-80e7-5c6bd7f0a6d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:57.796566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b92b05-04b6-4883-80e7-5c6bd7f0a6d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:57.798174 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b92b05-04b6-4883-80e7-5c6bd7f0a6d1 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:57.822066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 342/697] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:05:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:57.826218 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b92b05-04b6-4883-80e7-5c6bd7f0a6d1 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:57.942045 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:57.962118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b92b05-04b6-4883-80e7-5c6bd7f0a6d1 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:57.968015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a26c6f-645b-486e-beb9-a4c14ea1b0a6 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:58.010128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02cf72-c9e3-4520-ad0b-071669d09174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:58.010749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02cf72-c9e3-4520-ad0b-071669d09174 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:58.011030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02cf72-c9e3-4520-ad0b-071669d09174 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:58.011407 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02cf72-c9e3-4520-ad0b-071669d09174 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:58.012086 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02cf72-c9e3-4520-ad0b-071669d09174 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:58.012391 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02cf72-c9e3-4520-ad0b-071669d09174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:58.013782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02cf72-c9e3-4520-ad0b-071669d09174 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:58.028002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 356/698] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:05:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:58.034021 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02cf72-c9e3-4520-ad0b-071669d09174 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:58.040249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982209a-5806-4121-941a-ab695f1973d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:58.040874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982209a-5806-4121-941a-ab695f1973d9 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:58.041152 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982209a-5806-4121-941a-ab695f1973d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:58.041423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982209a-5806-4121-941a-ab695f1973d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:58.042158 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982209a-5806-4121-941a-ab695f1973d9 None admin] RBAC: Adding query filter params (role.id=28365abffd3d4ff6874bce2e0c561f1d, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=cf462b2aa3fc4e2ca58851549cf0c644) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:58.042470 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982209a-5806-4121-941a-ab695f1973d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:58.043948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982209a-5806-4121-941a-ab695f1973d9 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:58.062049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 343/699] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:05:57 2022] GET /identity/v3/role_assignments?user.id=cf462b2aa3fc4e2ca58851549cf0c644&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=28365abffd3d4ff6874bce2e0c561f1d => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:58.178001 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02cf72-c9e3-4520-ad0b-071669d09174 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:58.242846 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b11602-58e6-407d-b734-9de83ef82812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:58.243462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b11602-58e6-407d-b734-9de83ef82812 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:58.243746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b11602-58e6-407d-b734-9de83ef82812 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:58.244022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b11602-58e6-407d-b734-9de83ef82812 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:58.255526 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b11602-58e6-407d-b734-9de83ef82812 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:58.256994 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b11602-58e6-407d-b734-9de83ef82812 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:58.258921 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:58.277767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-a8b11602-58e6-407d-b734-9de83ef82812 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:58.277767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:58.277767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:58.277767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:58.277767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:58.277767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:58.277767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:58.277767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:58.277767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:58.277767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:58.277767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:58.277767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:58.277767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:58.277767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:58.277767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:58.277767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:58.278795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:58.278795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:58.278795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:58.278795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:58.278795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:58.278795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:58.278795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:58.278795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:58.278795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:58.278795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:58.278795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:58.278795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:58.278795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:58.278795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:58.278795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:58.278795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:58.278795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:58.279725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:58.279725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:58.279725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:58.284118 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 357/700] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:58.284697 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982209a-5806-4121-941a-ab695f1973d9 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:58.344474 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:58.402782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982209a-5806-4121-941a-ab695f1973d9 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:58.447387 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc49206-a62c-4117-8a38-af7b5303bc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:58.448348 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc49206-a62c-4117-8a38-af7b5303bc00 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:58.448863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc49206-a62c-4117-8a38-af7b5303bc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:58.449425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc49206-a62c-4117-8a38-af7b5303bc00 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:58.450273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc49206-a62c-4117-8a38-af7b5303bc00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:58.450780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc49206-a62c-4117-8a38-af7b5303bc00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:58.452408 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc49206-a62c-4117-8a38-af7b5303bc00 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:58.470046 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 344/701] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:58.475147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b11602-58e6-407d-b734-9de83ef82812 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:58.589129 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b11602-58e6-407d-b734-9de83ef82812 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:58.627366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39854a8c-4f58-4521-b526-e81df03e5b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:58.628018 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39854a8c-4f58-4521-b526-e81df03e5b30 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:58.628299 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39854a8c-4f58-4521-b526-e81df03e5b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:58.628569 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39854a8c-4f58-4521-b526-e81df03e5b30 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:58.637866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39854a8c-4f58-4521-b526-e81df03e5b30 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:58.639354 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39854a8c-4f58-4521-b526-e81df03e5b30 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:58.651754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-39854a8c-4f58-4521-b526-e81df03e5b30 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:05:58.651754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:58.651754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:58.651754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:58.651754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:58.651754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:58.651754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:58.651754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:58.651754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:58.651754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:58.651754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:58.651754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:58.651754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:05:58.651754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:05:58.651754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:05:58.651754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:58.652847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:58.653796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:58.653796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:58.653796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:05:58.653796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:05:58.653796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:05:58.653796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:05:58.653796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:05:58.653796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:05:58.653796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:05:58.653796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:58.653796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 358/702] 10.222.0.78 () {56 vars in 1093 bytes} [Mon Sep 19 11:05:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:58.659004 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc49206-a62c-4117-8a38-af7b5303bc00 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:58.780930 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc49206-a62c-4117-8a38-af7b5303bc00 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:58.806469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:58.834669 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048e28a-b2aa-41f0-aa79-1000d74d147b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:58.835313 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048e28a-b2aa-41f0-aa79-1000d74d147b None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:58.835588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048e28a-b2aa-41f0-aa79-1000d74d147b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:58.835903 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048e28a-b2aa-41f0-aa79-1000d74d147b None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:58.837201 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048e28a-b2aa-41f0-aa79-1000d74d147b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:58.838188 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048e28a-b2aa-41f0-aa79-1000d74d147b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:58.838916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048e28a-b2aa-41f0-aa79-1000d74d147b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:58.865453 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 345/703] 10.222.0.78 () {56 vars in 1133 bytes} [Mon Sep 19 11:05:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:58.871292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39854a8c-4f58-4521-b526-e81df03e5b30 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:58.950742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:59.003266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39854a8c-4f58-4521-b526-e81df03e5b30 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:59.038111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb6a3a-b885-4043-91c0-c108e50fb66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:59.038791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb6a3a-b885-4043-91c0-c108e50fb66a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:59.039192 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb6a3a-b885-4043-91c0-c108e50fb66a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:59.039523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb6a3a-b885-4043-91c0-c108e50fb66a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:59.136273 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb6a3a-b885-4043-91c0-c108e50fb66a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:59.138054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb6a3a-b885-4043-91c0-c108e50fb66a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:59.208672 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:59.247488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d4aa0-1cac-4899-b26a-0f89c8782b18 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:59.247808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d4aa0-1cac-4899-b26a-0f89c8782b18 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:59.248043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d4aa0-1cac-4899-b26a-0f89c8782b18 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:59.253523 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 346/704] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:59.254690 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868b1d1-ec7b-405d-8aa7-9c41ad9eb8ea None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:59.254967 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868b1d1-ec7b-405d-8aa7-9c41ad9eb8ea None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:59.255381 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868b1d1-ec7b-405d-8aa7-9c41ad9eb8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:59.261330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 359/705] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:05:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:05:59.267290 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb6a3a-b885-4043-91c0-c108e50fb66a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:59.293047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-4868b1d1-ec7b-405d-8aa7-9c41ad9eb8ea None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:59.294262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4868b1d1-ec7b-405d-8aa7-9c41ad9eb8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:59.372502 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb6a3a-b885-4043-91c0-c108e50fb66a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:59.377105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 347/706] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:05:59.378326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84122a-1537-4af2-a210-e9a5f28aa171 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:59.378610 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84122a-1537-4af2-a210-e9a5f28aa171 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:59.383885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84122a-1537-4af2-a210-e9a5f28aa171 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:59.385222 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 348/707] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:05:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:05:59.390687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7803553-6e3d-4b13-818c-2ca9c7f71794 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:59.390971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7803553-6e3d-4b13-818c-2ca9c7f71794 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:59.391278 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7803553-6e3d-4b13-818c-2ca9c7f71794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:59.418403 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-b7803553-6e3d-4b13-818c-2ca9c7f71794 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:05:59.420385 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b7803553-6e3d-4b13-818c-2ca9c7f71794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:05:59.439485 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ef53b-7ec2-4bdb-8f4e-116808018345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:59.440133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ef53b-7ec2-4bdb-8f4e-116808018345 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:59.440417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ef53b-7ec2-4bdb-8f4e-116808018345 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:59.440803 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ef53b-7ec2-4bdb-8f4e-116808018345 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:59.457784 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ef53b-7ec2-4bdb-8f4e-116808018345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:59.460365 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ef53b-7ec2-4bdb-8f4e-116808018345 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:59.487343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-9d6ef53b-7ec2-4bdb-8f4e-116808018345 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:59.487343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:59.487343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:59.487343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:59.487343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:59.487343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:59.487343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:59.487343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:59.487343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:59.487343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:59.487343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:59.487343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:59.487343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:05:59.487343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:05:59.487343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:05:59.487343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:05:59.488346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:59.488346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:59.488346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:59.488346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:59.488346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:59.488346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:59.488346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:59.488346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:59.488346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:59.488346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:59.488346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:59.488346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:59.488346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:59.488346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:59.488346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:59.488346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:59.488346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:05:59.489244 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:05:59.489244 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:05:59.489244 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:05:59.489244 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 360/708] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:05:59.495660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ef53b-7ec2-4bdb-8f4e-116808018345 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:59.516247 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:59.568821 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 349/709] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:05:59.611779 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ef53b-7ec2-4bdb-8f4e-116808018345 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:59.647465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55e9fb-6967-4bcb-aee0-b045661e8275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:59.648000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55e9fb-6967-4bcb-aee0-b045661e8275 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:59.648284 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55e9fb-6967-4bcb-aee0-b045661e8275 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:59.648593 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55e9fb-6967-4bcb-aee0-b045661e8275 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:59.649247 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55e9fb-6967-4bcb-aee0-b045661e8275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:05:59.649549 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55e9fb-6967-4bcb-aee0-b045661e8275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:59.650918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55e9fb-6967-4bcb-aee0-b045661e8275 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:59.663679 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 361/710] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:05:59.669342 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7803553-6e3d-4b13-818c-2ca9c7f71794 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:59.685193 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55e9fb-6967-4bcb-aee0-b045661e8275 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:05:59.724001 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:59.785840 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:05:59.824733 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55e9fb-6967-4bcb-aee0-b045661e8275 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:59.883375 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7803553-6e3d-4b13-818c-2ca9c7f71794 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:05:59.940724 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63594c-2e19-48f8-aa12-3c9b22aa5d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:05:59.941290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63594c-2e19-48f8-aa12-3c9b22aa5d94 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:05:59.941563 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63594c-2e19-48f8-aa12-3c9b22aa5d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:05:59.941830 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63594c-2e19-48f8-aa12-3c9b22aa5d94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:05:59.950148 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f63594c-2e19-48f8-aa12-3c9b22aa5d94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:05:59.951723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f63594c-2e19-48f8-aa12-3c9b22aa5d94 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:05:59.965816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-3f63594c-2e19-48f8-aa12-3c9b22aa5d94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:59.965816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:05:59.965816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:05:59.965816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:05:59.965816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:05:59.965816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:05:59.965816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:05:59.965816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:05:59.965816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:05:59.965816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:05:59.965816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:05:59.965816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:05:59.965816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:05:59.965816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:05:59.965816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:05:59.966783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:05:59.966783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:05:59.966783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:05:59.966783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:05:59.966783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:05:59.966783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:05:59.966783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:05:59.966783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:05:59.966783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:05:59.966783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:05:59.966783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:05:59.966783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:05:59.966783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:05:59.966783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:05:59.966783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:05:59.966783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:05:59.966783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:05:59.967723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:05:59.967723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:05:59.967723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:05:59.967723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:05:59.967723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:05:59.967723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:05:59.967723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:05:59.967723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:05:59.967723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 350/711] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:05:59.968684 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63594c-2e19-48f8-aa12-3c9b22aa5d94 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:00.031462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c560d3d-a1c5-4890-b5da-bbf1a33651b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:00.032034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c560d3d-a1c5-4890-b5da-bbf1a33651b8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:00.032402 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c560d3d-a1c5-4890-b5da-bbf1a33651b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:00.032742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c560d3d-a1c5-4890-b5da-bbf1a33651b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:00.047932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c560d3d-a1c5-4890-b5da-bbf1a33651b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:00.049542 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c560d3d-a1c5-4890-b5da-bbf1a33651b8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:00.068154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-1c560d3d-a1c5-4890-b5da-bbf1a33651b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:00.068154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:00.068154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:00.068154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:00.068154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:00.068154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:00.068154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:00.068154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:00.068154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:00.068154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:00.068154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:00.068154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:00.068154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:00.068154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:00.068154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:00.068154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:00.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:00.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:00.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:00.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:00.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:00.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:00.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:00.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:00.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:00.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:00.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:00.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:00.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:00.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:00.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:00.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:00.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:00.070085 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:00.070085 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:00.070085 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:00.075103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 362/712] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:00.076672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:00.077668 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c560d3d-a1c5-4890-b5da-bbf1a33651b8 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:00.093839 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63594c-2e19-48f8-aa12-3c9b22aa5d94 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:00.142475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc680d3b-a215-4023-9115-46b241cf5e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:00.143388 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc680d3b-a215-4023-9115-46b241cf5e39 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:00.143878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc680d3b-a215-4023-9115-46b241cf5e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:00.144291 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc680d3b-a215-4023-9115-46b241cf5e39 None admin] PATH_INFO: `/v3/services/network` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:00.145042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc680d3b-a215-4023-9115-46b241cf5e39 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:00.146546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc680d3b-a215-4023-9115-46b241cf5e39 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:00.156410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-dc680d3b-a215-4023-9115-46b241cf5e39 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:06:00.156410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:00.156410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:00.156410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:00.156410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:00.156410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:00.156410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:00.156410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:00.156410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:06:00.156410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:00.156410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:00.156410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:00.156410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:00.156410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:00.156410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:00.156410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:00.156410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:00.157409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:00.157409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:00.157409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:00.157409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:00.157409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:00.157409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:00.157409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:00.157409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:00.157409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:00.157409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:00.157409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:00.157409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:00.157409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:00.157409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:00.157409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:00.157409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:00.158328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:00.158328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:00.158328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:00.158328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:00.158328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:00.158328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:00.158328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:00.158328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:00.158328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:00.158328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:00.158328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:00.158328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:00.158328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:06:00.158328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:00.160736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 351/713] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:05:59 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:00.162189 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501f4ff-08d1-4d8b-8858-ab5bad8c09e7 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:00.162466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501f4ff-08d1-4d8b-8858-ab5bad8c09e7 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:00.162737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501f4ff-08d1-4d8b-8858-ab5bad8c09e7 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:00.167212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 352/714] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:00.167212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501f4ff-08d1-4d8b-8858-ab5bad8c09e7 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:00.198561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c560d3d-a1c5-4890-b5da-bbf1a33651b8 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:00.221780 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:00.243762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca3b44-35e6-46fd-9778-2e47db5cb220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:00.244821 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca3b44-35e6-46fd-9778-2e47db5cb220 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:00.245109 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca3b44-35e6-46fd-9778-2e47db5cb220 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:00.245382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca3b44-35e6-46fd-9778-2e47db5cb220 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:00.246508 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca3b44-35e6-46fd-9778-2e47db5cb220 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:00.246824 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca3b44-35e6-46fd-9778-2e47db5cb220 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:00.248941 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca3b44-35e6-46fd-9778-2e47db5cb220 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:00.262778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 363/715] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:06:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:00.263694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca3b44-35e6-46fd-9778-2e47db5cb220 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:00.274241 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501f4ff-08d1-4d8b-8858-ab5bad8c09e7 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:00.308948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d55f71-0a3f-4881-b85b-a3b338c80b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:00.310239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d55f71-0a3f-4881-b85b-a3b338c80b05 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:00.310520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d55f71-0a3f-4881-b85b-a3b338c80b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:00.310793 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d55f71-0a3f-4881-b85b-a3b338c80b05 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:00.311471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d55f71-0a3f-4881-b85b-a3b338c80b05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:00.311772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d55f71-0a3f-4881-b85b-a3b338c80b05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:00.313165 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d55f71-0a3f-4881-b85b-a3b338c80b05 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:00.325947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 353/716] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:00.326926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702da223-0706-4350-a220-a2301789be2a None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:00.327244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702da223-0706-4350-a220-a2301789be2a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:00.327523 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702da223-0706-4350-a220-a2301789be2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:00.347857 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-702da223-0706-4350-a220-a2301789be2a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:00.349045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-702da223-0706-4350-a220-a2301789be2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:00.352686 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:00.373525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca3b44-35e6-46fd-9778-2e47db5cb220 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:00.413074 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2529a9-9604-4547-ba9b-8e5c39fc36cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:00.413692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2529a9-9604-4547-ba9b-8e5c39fc36cb None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:00.414025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2529a9-9604-4547-ba9b-8e5c39fc36cb None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:00.414324 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2529a9-9604-4547-ba9b-8e5c39fc36cb None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:00.414930 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2529a9-9604-4547-ba9b-8e5c39fc36cb None admin] RBAC: Adding query filter params (name=network) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:00.415473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2529a9-9604-4547-ba9b-8e5c39fc36cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:00.416621 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2529a9-9604-4547-ba9b-8e5c39fc36cb None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:00.420714 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 354/717] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:06:00.421328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702da223-0706-4350-a220-a2301789be2a None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:00.432075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 364/718] 10.222.0.78 () {56 vars in 1112 bytes} [Mon Sep 19 11:06:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:00.433523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2529a9-9604-4547-ba9b-8e5c39fc36cb None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:00.496836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:00.533506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702da223-0706-4350-a220-a2301789be2a None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:00.574031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c7e3c-c80e-4592-8f91-dd83f988e601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:00.574638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c7e3c-c80e-4592-8f91-dd83f988e601 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:00.575004 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c7e3c-c80e-4592-8f91-dd83f988e601 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:00.575396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c7e3c-c80e-4592-8f91-dd83f988e601 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:00.576181 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c7e3c-c80e-4592-8f91-dd83f988e601 None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=aba1968529574fe5b4c5d114c874f428) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:00.576583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c7e3c-c80e-4592-8f91-dd83f988e601 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:00.578087 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c7e3c-c80e-4592-8f91-dd83f988e601 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:00.578433 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2529a9-9604-4547-ba9b-8e5c39fc36cb None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:00.584336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 355/719] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:06:00 2022] GET /identity/v3/role_assignments?user.id=aba1968529574fe5b4c5d114c874f428&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:00.585043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c7e3c-c80e-4592-8f91-dd83f988e601 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:00.590075 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:00.632077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449b74d-e892-4d6e-80fb-c3bc3c831b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:00.633606 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449b74d-e892-4d6e-80fb-c3bc3c831b38 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:00.634551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449b74d-e892-4d6e-80fb-c3bc3c831b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:00.635014 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449b74d-e892-4d6e-80fb-c3bc3c831b38 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:00.636317 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449b74d-e892-4d6e-80fb-c3bc3c831b38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:00.638262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449b74d-e892-4d6e-80fb-c3bc3c831b38 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:00.705859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c7e3c-c80e-4592-8f91-dd83f988e601 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:00.744304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 365/720] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:06:00 2022] POST /identity/v3/users => generated 254 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:06:00.745164 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449b74d-e892-4d6e-80fb-c3bc3c831b38 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:00.751375 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b5733-70f8-4a7b-9a9f-95dcdaaff02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:00.752378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b5733-70f8-4a7b-9a9f-95dcdaaff02f None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:00.752665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b5733-70f8-4a7b-9a9f-95dcdaaff02f None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:00.752938 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b5733-70f8-4a7b-9a9f-95dcdaaff02f None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:00.753583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b5733-70f8-4a7b-9a9f-95dcdaaff02f None admin] RBAC: Adding query filter params (type=network) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:00.754003 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b5733-70f8-4a7b-9a9f-95dcdaaff02f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:00.755584 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b5733-70f8-4a7b-9a9f-95dcdaaff02f None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:00.770234 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 356/721] 10.222.0.78 () {56 vars in 1112 bytes} [Mon Sep 19 11:06:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:00.911398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:00.917895 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449b74d-e892-4d6e-80fb-c3bc3c831b38 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:00.968663 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084646d-df3b-47e8-b7f8-b35fce4a9f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:00.969295 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084646d-df3b-47e8-b7f8-b35fce4a9f05 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:00.969577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084646d-df3b-47e8-b7f8-b35fce4a9f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:00.969851 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084646d-df3b-47e8-b7f8-b35fce4a9f05 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:00.970516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084646d-df3b-47e8-b7f8-b35fce4a9f05 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:00.971936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084646d-df3b-47e8-b7f8-b35fce4a9f05 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:01.080819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 366/722] 10.222.0.78 () {60 vars in 1141 bytes} [Mon Sep 19 11:06:00 2022] POST /identity/v3/services => generated 251 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:06:01.142182 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c44d46-4f36-4161-9e5b-4506969c962a None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:01.142878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c44d46-4f36-4161-9e5b-4506969c962a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:01.143212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c44d46-4f36-4161-9e5b-4506969c962a None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:01.145193 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 357/723] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:01.150969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28489c8c-7ae0-4349-a670-c359f98152ca None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:01.151299 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28489c8c-7ae0-4349-a670-c359f98152ca None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:01.151560 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28489c8c-7ae0-4349-a670-c359f98152ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:01.175614 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-28489c8c-7ae0-4349-a670-c359f98152ca None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:01.176879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-28489c8c-7ae0-4349-a670-c359f98152ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:01.291668 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 367/724] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:06:01.623727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c44d46-4f36-4161-9e5b-4506969c962a None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:01.704291 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:01.761279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c44d46-4f36-4161-9e5b-4506969c962a None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:01.808220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa32a7-8ccf-4d42-9d6c-2ef86e05b5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:01.808871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa32a7-8ccf-4d42-9d6c-2ef86e05b5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:01.809149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa32a7-8ccf-4d42-9d6c-2ef86e05b5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:01.809421 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa32a7-8ccf-4d42-9d6c-2ef86e05b5e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:01.825292 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa32a7-8ccf-4d42-9d6c-2ef86e05b5e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:06:01.830679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-edaa32a7-8ccf-4d42-9d6c-2ef86e05b5e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:06:01.830679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:01.830679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:01.830679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:01.830679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:01.830679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:01.830679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:01.830679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:01.830679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:01.830679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:01.830679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:01.830679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:01.830679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:06:01.830679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:06:01.830679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:06:01.830679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:06:01.831638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:06:01.831638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:06:01.831638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:06:01.831638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:06:01.831638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:01.831638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:01.831638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:01.831638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:01.831638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:01.831638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:01.831638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:01.831638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:01.831638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:01.831638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:01.831638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:01.831638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:01.832462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:01.832462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:01.832462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:01.832462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:01.832462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:06:01.832462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:06:01.832462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:06:01.832462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:06:01.832462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:06:01.832462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:06:01.832462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:06:01.832462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:01.832462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 358/725] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:06:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:01.838928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28489c8c-7ae0-4349-a670-c359f98152ca None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:01.966992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:01.984920 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28489c8c-7ae0-4349-a670-c359f98152ca None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:02.030666 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1df349-7f1f-48a9-9db7-9d8c562bdf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:02.031423 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1df349-7f1f-48a9-9db7-9d8c562bdf0a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:02.031699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1df349-7f1f-48a9-9db7-9d8c562bdf0a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:02.031978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1df349-7f1f-48a9-9db7-9d8c562bdf0a None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:02.032671 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1df349-7f1f-48a9-9db7-9d8c562bdf0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:02.032980 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1df349-7f1f-48a9-9db7-9d8c562bdf0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:02.034387 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1df349-7f1f-48a9-9db7-9d8c562bdf0a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:02.050385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 368/726] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:06:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:02.056742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa32a7-8ccf-4d42-9d6c-2ef86e05b5e3 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:02.223128 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa32a7-8ccf-4d42-9d6c-2ef86e05b5e3 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:02.265902 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:02.275884 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16f3cd-8784-4ab7-8940-070d4fc6cf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:02.280051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16f3cd-8784-4ab7-8940-070d4fc6cf3f None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:02.280051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16f3cd-8784-4ab7-8940-070d4fc6cf3f None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:02.280051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16f3cd-8784-4ab7-8940-070d4fc6cf3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:02.290760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16f3cd-8784-4ab7-8940-070d4fc6cf3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:02.292242 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16f3cd-8784-4ab7-8940-070d4fc6cf3f None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:02.312042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-af16f3cd-8784-4ab7-8940-070d4fc6cf3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:02.312042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:02.312042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:02.312042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:02.312042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:02.312042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:02.312042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:02.312042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:02.312042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:02.312042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:02.312042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:02.312042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:02.312042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:02.312042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:02.312042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:02.312042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:02.313056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:02.313056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:02.313056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:02.313056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:02.313056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:02.313056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:02.313056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:02.313056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:02.313056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:02.313056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:02.313056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:02.313056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:02.313056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:02.313056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:02.313056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:02.313056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:02.313056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:02.313990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:02.313990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:02.313990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:02.313990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 359/727] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:02.320839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1df349-7f1f-48a9-9db7-9d8c562bdf0a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:02.470011 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1df349-7f1f-48a9-9db7-9d8c562bdf0a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:02.516898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a682af-1c89-44f9-8022-9ece46d12126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:02.517917 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a682af-1c89-44f9-8022-9ece46d12126 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:02.518403 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a682af-1c89-44f9-8022-9ece46d12126 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:02.518901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a682af-1c89-44f9-8022-9ece46d12126 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:02.519840 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a682af-1c89-44f9-8022-9ece46d12126 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:02.520347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a682af-1c89-44f9-8022-9ece46d12126 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:02.521938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a682af-1c89-44f9-8022-9ece46d12126 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:02.538227 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 369/728] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:02.548331 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16f3cd-8784-4ab7-8940-070d4fc6cf3f None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:02.691748 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16f3cd-8784-4ab7-8940-070d4fc6cf3f None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:02.739934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa32697-43d1-4588-9078-4e334f7dae5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:02.740577 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa32697-43d1-4588-9078-4e334f7dae5e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:02.740860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa32697-43d1-4588-9078-4e334f7dae5e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:02.741138 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa32697-43d1-4588-9078-4e334f7dae5e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:02.753552 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa32697-43d1-4588-9078-4e334f7dae5e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:02.754996 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa32697-43d1-4588-9078-4e334f7dae5e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:02.769277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-9aa32697-43d1-4588-9078-4e334f7dae5e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:06:02.769277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:02.769277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:02.769277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:02.769277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:02.769277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:02.769277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:02.769277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:02.769277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:02.769277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:02.769277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:02.769277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:02.769277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:06:02.769277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:06:02.769277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:06:02.769277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:02.770235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:02.771187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:02.771187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:02.771187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:06:02.771187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:06:02.771187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:06:02.771187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:06:02.771187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:06:02.771187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:06:02.771187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:06:02.771187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:02.775278 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 360/729] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:06:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:02.777135 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a682af-1c89-44f9-8022-9ece46d12126 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:02.799529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:02.942990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a682af-1c89-44f9-8022-9ece46d12126 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:03.001669 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1dc28-c48f-4f1a-bb8c-95e8d822a51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:03.002291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1dc28-c48f-4f1a-bb8c-95e8d822a51a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:03.002563 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1dc28-c48f-4f1a-bb8c-95e8d822a51a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:03.002828 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1dc28-c48f-4f1a-bb8c-95e8d822a51a None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:03.003583 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1dc28-c48f-4f1a-bb8c-95e8d822a51a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:03.003883 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1dc28-c48f-4f1a-bb8c-95e8d822a51a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:03.005297 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1dc28-c48f-4f1a-bb8c-95e8d822a51a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:03.034033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 370/730] 10.222.0.78 () {56 vars in 1135 bytes} [Mon Sep 19 11:06:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:03.042047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa32697-43d1-4588-9078-4e334f7dae5e None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:03.098622 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:03.170979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa32697-43d1-4588-9078-4e334f7dae5e None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:03.213031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca207bb-1a6c-440b-b4d2-19dff00d74c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:03.213649 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca207bb-1a6c-440b-b4d2-19dff00d74c9 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:03.213927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca207bb-1a6c-440b-b4d2-19dff00d74c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:03.214198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca207bb-1a6c-440b-b4d2-19dff00d74c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:03.287943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca207bb-1a6c-440b-b4d2-19dff00d74c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:03.289340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca207bb-1a6c-440b-b4d2-19dff00d74c9 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:03.339351 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:03.427306 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 361/731] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:06:03 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:06:03.429233 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1dc28-c48f-4f1a-bb8c-95e8d822a51a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:03.514536 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:03.530271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a5f6c-78de-4dcf-9b4d-cd05e0b69304 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:03.530906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a5f6c-78de-4dcf-9b4d-cd05e0b69304 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:03.531466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a5f6c-78de-4dcf-9b4d-cd05e0b69304 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:03.533325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 362/732] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:06:03.538877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115015e5-11f6-47ac-8173-8fc0a28851f0 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:03.539947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115015e5-11f6-47ac-8173-8fc0a28851f0 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:03.540439 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115015e5-11f6-47ac-8173-8fc0a28851f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:03.560645 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-115015e5-11f6-47ac-8173-8fc0a28851f0 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:03.562228 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-115015e5-11f6-47ac-8173-8fc0a28851f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:03.573591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1dc28-c48f-4f1a-bb8c-95e8d822a51a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:03.644372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f912ea-bfa2-46ec-9c72-f2ad3fe0d1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:03.645127 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f912ea-bfa2-46ec-9c72-f2ad3fe0d1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:03.645638 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f912ea-bfa2-46ec-9c72-f2ad3fe0d1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:03.646097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f912ea-bfa2-46ec-9c72-f2ad3fe0d1e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:03.657818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f912ea-bfa2-46ec-9c72-f2ad3fe0d1e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:03.659751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f912ea-bfa2-46ec-9c72-f2ad3fe0d1e6 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:03.674564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 363/733] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:06:03.675872 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cb3d7-24cd-415e-8dd4-ff9c939fc4af None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:03.676426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cb3d7-24cd-415e-8dd4-ff9c939fc4af None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:03.676827 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cb3d7-24cd-415e-8dd4-ff9c939fc4af None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:03.678783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 364/734] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:03.680911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-89f912ea-bfa2-46ec-9c72-f2ad3fe0d1e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:03.680911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:03.680911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:03.680911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:03.680911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:03.680911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:03.680911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:03.680911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:03.680911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:03.680911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:03.680911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:03.680911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:03.680911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:03.680911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:03.680911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:03.680911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:03.681829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:03.681829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:03.681829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:03.681829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:03.681829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:03.681829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:03.681829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:03.681829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:03.681829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:03.681829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:03.681829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:03.681829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:03.681829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:03.681829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:03.681829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:03.681829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:03.681829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:03.682798 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:03.682798 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:03.682798 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:03.688843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 371/735] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:03.688998 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fc37b-af10-4359-9c15-6916e55ec080 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:03.693504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fc37b-af10-4359-9c15-6916e55ec080 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:03.693504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fc37b-af10-4359-9c15-6916e55ec080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:03.695686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f912ea-bfa2-46ec-9c72-f2ad3fe0d1e6 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:03.712061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-f88fc37b-af10-4359-9c15-6916e55ec080 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:03.713677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f88fc37b-af10-4359-9c15-6916e55ec080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:03.792260 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:03.798972 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 365/736] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:06:03.806755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e5efd-5ec4-4be5-af86-eb50cb402b88 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:03.807383 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e5efd-5ec4-4be5-af86-eb50cb402b88 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:03.807843 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e5efd-5ec4-4be5-af86-eb50cb402b88 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:03.809676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 366/737] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:03 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:03.817302 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55b815-e6dd-4102-958b-2e1f451acb02 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:03.817882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55b815-e6dd-4102-958b-2e1f451acb02 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:03.818360 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55b815-e6dd-4102-958b-2e1f451acb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:03.850522 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-5f55b815-e6dd-4102-958b-2e1f451acb02 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:03.852639 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5f55b815-e6dd-4102-958b-2e1f451acb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:03.855528 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f912ea-bfa2-46ec-9c72-f2ad3fe0d1e6 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:03.875737 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:03.899981 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d611a-ea85-4b55-945f-2b13ffb9d634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:03.901002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d611a-ea85-4b55-945f-2b13ffb9d634 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:03.901488 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d611a-ea85-4b55-945f-2b13ffb9d634 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:03.902688 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d611a-ea85-4b55-945f-2b13ffb9d634 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:03.903787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d611a-ea85-4b55-945f-2b13ffb9d634 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:03.904307 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d611a-ea85-4b55-945f-2b13ffb9d634 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:03.906126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d611a-ea85-4b55-945f-2b13ffb9d634 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:03.926171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 372/738] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:03.934711 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d611a-ea85-4b55-945f-2b13ffb9d634 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:03.994988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 367/739] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:06:03.996389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55b815-e6dd-4102-958b-2e1f451acb02 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:04.066656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d611a-ea85-4b55-945f-2b13ffb9d634 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:04.105256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbb149-76ee-4ba4-bc8f-00ea9839bf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:04.105910 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbb149-76ee-4ba4-bc8f-00ea9839bf86 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:04.106194 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbb149-76ee-4ba4-bc8f-00ea9839bf86 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:04.106481 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbb149-76ee-4ba4-bc8f-00ea9839bf86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:04.112254 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55b815-e6dd-4102-958b-2e1f451acb02 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:04.116382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbb149-76ee-4ba4-bc8f-00ea9839bf86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:04.118417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbb149-76ee-4ba4-bc8f-00ea9839bf86 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:04.130998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-93cbb149-76ee-4ba4-bc8f-00ea9839bf86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:04.130998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:04.130998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:04.130998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:04.130998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:04.130998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:04.130998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:04.130998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:04.130998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:04.130998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:04.130998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:04.130998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:04.130998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:06:04.130998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:06:04.130998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:06:04.131950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:06:04.131950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:04.131950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:04.131950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:04.131950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:04.131950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:04.131950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:04.131950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:04.131950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:04.131950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:04.131950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:04.131950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:04.131950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:04.131950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:04.131950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:04.131950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:04.131950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:04.132902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:06:04.132902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:06:04.132902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:06:04.132902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:06:04.132902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:06:04.132902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:06:04.132902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:04.132902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:04.132902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 373/740] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:04.133414 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26fde6-80d8-4cce-8c1e-d46da0d47948 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:04.134198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26fde6-80d8-4cce-8c1e-d46da0d47948 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:04.134198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26fde6-80d8-4cce-8c1e-d46da0d47948 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:04.139360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 374/741] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:04.140744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30607639-7d47-4cf0-8ab5-a06baeea45b5 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:04.141078 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30607639-7d47-4cf0-8ab5-a06baeea45b5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:04.141396 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30607639-7d47-4cf0-8ab5-a06baeea45b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:04.157391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:04.162003 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-30607639-7d47-4cf0-8ab5-a06baeea45b5 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:04.163241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-30607639-7d47-4cf0-8ab5-a06baeea45b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:04.164987 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a1f51-add2-425c-b359-79a0aba6cb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:04.166009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a1f51-add2-425c-b359-79a0aba6cb3b None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:04.166498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a1f51-add2-425c-b359-79a0aba6cb3b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:04.167030 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a1f51-add2-425c-b359-79a0aba6cb3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:04.180549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a1f51-add2-425c-b359-79a0aba6cb3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:06:04.186394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-894a1f51-add2-425c-b359-79a0aba6cb3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:06:04.186394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:04.186394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:04.186394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:04.186394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:04.186394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:04.186394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:04.186394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:04.186394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:04.186394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:04.186394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:04.186394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:04.186394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:06:04.186394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:06:04.186394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:06:04.186394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:06:04.187362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:06:04.187362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:06:04.187362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:06:04.187362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:06:04.187362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:04.187362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:04.187362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:04.187362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:04.187362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:04.187362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:04.187362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:04.187362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:04.187362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:04.187362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:04.187362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:04.187362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:04.188259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:04.188259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:04.188259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:04.188259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:04.188259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:06:04.188259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:06:04.188259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:06:04.188259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:06:04.188259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:06:04.188259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:06:04.188259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:06:04.188259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:04.191087 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 368/742] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:06:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:04.192283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a1f51-add2-425c-b359-79a0aba6cb3b None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:04.200734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:04.243007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 375/743] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:04 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:06:04.243707 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30607639-7d47-4cf0-8ab5-a06baeea45b5 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:04.328181 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a1f51-add2-425c-b359-79a0aba6cb3b None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:04.355330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30607639-7d47-4cf0-8ab5-a06baeea45b5 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:04.374047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df7b96-08f4-4a92-83ab-b4173dde0a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:04.374657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df7b96-08f4-4a92-83ab-b4173dde0a92 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:04.374940 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df7b96-08f4-4a92-83ab-b4173dde0a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:04.375237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df7b96-08f4-4a92-83ab-b4173dde0a92 None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:04.376555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df7b96-08f4-4a92-83ab-b4173dde0a92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:04.376555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df7b96-08f4-4a92-83ab-b4173dde0a92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:04.377923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df7b96-08f4-4a92-83ab-b4173dde0a92 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:04.390134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9b706-5bc8-47f6-baca-92b929c79467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:04.390388 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 369/744] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:06:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:04.391116 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9b706-5bc8-47f6-baca-92b929c79467 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:04.391485 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9b706-5bc8-47f6-baca-92b929c79467 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:04.391763 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9b706-5bc8-47f6-baca-92b929c79467 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:04.396434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df7b96-08f4-4a92-83ab-b4173dde0a92 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:04.403288 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9b706-5bc8-47f6-baca-92b929c79467 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:06:04.408103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-9ba9b706-5bc8-47f6-baca-92b929c79467 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:06:04.408103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:04.408103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:04.408103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:04.408103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:04.408103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:04.408103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:04.408103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:04.408103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:04.408103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:04.408103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:04.408103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:04.408103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:06:04.408103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:06:04.408103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:06:04.408103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:06:04.409125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:06:04.409125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:06:04.409125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:06:04.409125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:06:04.409125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:04.409125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:04.409125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:04.409125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:04.409125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:04.409125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:04.409125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:04.409125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:04.409125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:04.409125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:04.409125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:04.409125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:04.409969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:04.409969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:04.409969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:04.409969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:04.409969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:06:04.409969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:06:04.409969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:06:04.409969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:06:04.409969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:06:04.409969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:06:04.409969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:06:04.409969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:04.414784 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 376/745] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:06:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:04.415473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9b706-5bc8-47f6-baca-92b929c79467 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:04.444173 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:04.454156 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:04.506374 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df7b96-08f4-4a92-83ab-b4173dde0a92 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:04.519350 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9b706-5bc8-47f6-baca-92b929c79467 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:04.542332 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb001b9-7909-4b33-aab5-d9e12661c370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:04.542943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb001b9-7909-4b33-aab5-d9e12661c370 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:04.543251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb001b9-7909-4b33-aab5-d9e12661c370 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:04.543532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb001b9-7909-4b33-aab5-d9e12661c370 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:04.544188 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb001b9-7909-4b33-aab5-d9e12661c370 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:04.544496 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb001b9-7909-4b33-aab5-d9e12661c370 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:04.545898 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb001b9-7909-4b33-aab5-d9e12661c370 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:04.554084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd787aa-cb3b-4886-bc6e-5937bfc9e490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:04.554248 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 370/746] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:06:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:04.554702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd787aa-cb3b-4886-bc6e-5937bfc9e490 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:04.554983 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd787aa-cb3b-4886-bc6e-5937bfc9e490 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:04.555080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb001b9-7909-4b33-aab5-d9e12661c370 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:04.555294 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd787aa-cb3b-4886-bc6e-5937bfc9e490 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:04.555945 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd787aa-cb3b-4886-bc6e-5937bfc9e490 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:04.557303 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd787aa-cb3b-4886-bc6e-5937bfc9e490 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:04.650908 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 377/747] 10.222.0.78 () {60 vars in 1141 bytes} [Mon Sep 19 11:06:04 2022] POST /identity/v3/services => generated 240 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:06:04.651833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd787aa-cb3b-4886-bc6e-5937bfc9e490 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:04.662591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb001b9-7909-4b33-aab5-d9e12661c370 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:04.678546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:04.708373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53d279-aab7-4e68-bd67-9578eb3929a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:04.709087 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53d279-aab7-4e68-bd67-9578eb3929a4 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:04.709451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53d279-aab7-4e68-bd67-9578eb3929a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:04.709791 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53d279-aab7-4e68-bd67-9578eb3929a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:04.711360 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53d279-aab7-4e68-bd67-9578eb3929a4 None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=cf462b2aa3fc4e2ca58851549cf0c644) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:04.711830 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53d279-aab7-4e68-bd67-9578eb3929a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:04.713409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53d279-aab7-4e68-bd67-9578eb3929a4 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:04.730529 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 371/748] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:06:04 2022] GET /identity/v3/role_assignments?user.id=cf462b2aa3fc4e2ca58851549cf0c644&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:04.730529 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53d279-aab7-4e68-bd67-9578eb3929a4 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:04.778062 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:04.800756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd787aa-cb3b-4886-bc6e-5937bfc9e490 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:04.842474 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0665e7-195b-4d2f-9086-6f663c97e924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:04.843204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0665e7-195b-4d2f-9086-6f663c97e924 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:04.843793 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0665e7-195b-4d2f-9086-6f663c97e924 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:04.844093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0665e7-195b-4d2f-9086-6f663c97e924 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:04.844953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0665e7-195b-4d2f-9086-6f663c97e924 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:04.845365 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0665e7-195b-4d2f-9086-6f663c97e924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:04.847220 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0665e7-195b-4d2f-9086-6f663c97e924 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:04.859996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 378/749] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:06:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:04.861304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0665e7-195b-4d2f-9086-6f663c97e924 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:04.872105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53d279-aab7-4e68-bd67-9578eb3929a4 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:04.917823 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c5a9d-a88d-4856-b41d-d5a5cc1faee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:04.918489 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c5a9d-a88d-4856-b41d-d5a5cc1faee1 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:04.918771 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c5a9d-a88d-4856-b41d-d5a5cc1faee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:04.919049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c5a9d-a88d-4856-b41d-d5a5cc1faee1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:04.919668 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c5a9d-a88d-4856-b41d-d5a5cc1faee1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:04.921042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c5a9d-a88d-4856-b41d-d5a5cc1faee1 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:04.931343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-7a4c5a9d-a88d-4856-b41d-d5a5cc1faee1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:06:04.931343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:04.931343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:04.931343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:04.931343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:04.931343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:04.931343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:04.931343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:04.931343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:06:04.931343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:04.931343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:04.931343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:04.931343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:04.931343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:04.931343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:04.931343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:04.931343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:04.932586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:04.932586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:04.932586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:04.932586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:04.932586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:04.932586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:04.932586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:04.932586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:04.932586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:04.932586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:04.932586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:04.932586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:04.932586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:04.932586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:04.932586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:04.932586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:04.933478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:04.933478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:04.933478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:04.933478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:04.933478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:04.933478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:04.933478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:04.933478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:04.933478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:04.933478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:04.933478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:04.933478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:04.933478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:06:04.933478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:04.937357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 372/750] 10.222.0.78 () {56 vars in 1129 bytes} [Mon Sep 19 11:06:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:04.938242 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c5a9d-a88d-4856-b41d-d5a5cc1faee1 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:04.976058 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:05.000218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0665e7-195b-4d2f-9086-6f663c97e924 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:05.043343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7708c-a5ee-4b18-94ca-6a9267b9aa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:05.043991 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7708c-a5ee-4b18-94ca-6a9267b9aa8b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:05.044275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7708c-a5ee-4b18-94ca-6a9267b9aa8b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:05.044550 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7708c-a5ee-4b18-94ca-6a9267b9aa8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:05.053613 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c5a9d-a88d-4856-b41d-d5a5cc1faee1 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:05.056673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7708c-a5ee-4b18-94ca-6a9267b9aa8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:05.058140 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7708c-a5ee-4b18-94ca-6a9267b9aa8b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:05.066917 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:05.074623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-a5b7708c-a5ee-4b18-94ca-6a9267b9aa8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:05.074623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:05.074623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:05.074623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:05.074623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:05.074623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:05.074623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:05.074623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:05.074623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:05.074623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:05.074623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:05.074623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:05.074623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:05.074623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:05.074623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:05.074623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:05.083034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:05.083034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:05.083034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:05.083034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:05.083034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:05.083034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:05.083034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:05.083034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:05.083034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:05.083034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:05.083034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:05.083034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:05.083034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:05.083034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:05.083034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:05.083034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:05.083034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:05.084196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:05.084196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:05.084196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:05.084196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 379/751] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:05.084196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7708c-a5ee-4b18-94ca-6a9267b9aa8b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:05.092624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4333f0e-8dae-4207-a37c-b2c2ca88308c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:05.093140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4333f0e-8dae-4207-a37c-b2c2ca88308c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:05.093417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4333f0e-8dae-4207-a37c-b2c2ca88308c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:05.093738 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4333f0e-8dae-4207-a37c-b2c2ca88308c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:05.107061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4333f0e-8dae-4207-a37c-b2c2ca88308c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:05.108736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4333f0e-8dae-4207-a37c-b2c2ca88308c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:05.134782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-d4333f0e-8dae-4207-a37c-b2c2ca88308c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:05.134782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:05.134782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:05.134782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:05.134782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:05.134782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:05.134782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:05.134782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:05.134782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:05.134782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:05.134782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:05.134782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:05.134782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:05.134782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:05.134782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:05.134782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:05.135926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:05.135926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:05.135926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:05.135926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:05.135926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:05.135926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:05.135926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:05.135926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:05.135926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:05.135926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:05.135926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:05.135926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:05.135926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:05.135926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:05.135926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:05.135926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:05.135926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:05.137156 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:05.137156 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:05.137156 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:05.141303 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 373/752] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:05.142351 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4333f0e-8dae-4207-a37c-b2c2ca88308c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:05.193742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7708c-a5ee-4b18-94ca-6a9267b9aa8b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:05.231463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d7e80-810f-4712-bbdc-917512bea5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:05.232468 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d7e80-810f-4712-bbdc-917512bea5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:05.232934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d7e80-810f-4712-bbdc-917512bea5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:05.233431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d7e80-810f-4712-bbdc-917512bea5b8 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:05.234252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d7e80-810f-4712-bbdc-917512bea5b8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:05.234762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d7e80-810f-4712-bbdc-917512bea5b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:05.236447 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d7e80-810f-4712-bbdc-917512bea5b8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:05.241625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:05.247065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 380/753] 10.222.0.78 () {56 vars in 1124 bytes} [Mon Sep 19 11:06:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:05.247065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d7e80-810f-4712-bbdc-917512bea5b8 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:05.269453 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4333f0e-8dae-4207-a37c-b2c2ca88308c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:05.322858 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffffd7-2953-42e8-8daf-45784c83c430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:05.324165 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffffd7-2953-42e8-8daf-45784c83c430 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:05.324969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:05.325916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffffd7-2953-42e8-8daf-45784c83c430 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:05.326636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffffd7-2953-42e8-8daf-45784c83c430 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:05.327581 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffffd7-2953-42e8-8daf-45784c83c430 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:05.328112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffffd7-2953-42e8-8daf-45784c83c430 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:05.329966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffffd7-2953-42e8-8daf-45784c83c430 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:05.346180 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 374/754] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:05.347045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffffd7-2953-42e8-8daf-45784c83c430 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:05.370403 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d7e80-810f-4712-bbdc-917512bea5b8 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:05.407485 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ff247-5b88-4549-8e3f-afe9b73485fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:05.408113 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ff247-5b88-4549-8e3f-afe9b73485fc None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:05.408397 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ff247-5b88-4549-8e3f-afe9b73485fc None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:05.408676 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ff247-5b88-4549-8e3f-afe9b73485fc None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:05.409330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ff247-5b88-4549-8e3f-afe9b73485fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:05.409738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ff247-5b88-4549-8e3f-afe9b73485fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:05.411228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ff247-5b88-4549-8e3f-afe9b73485fc None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:05.426696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 381/755] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:05.427748 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ff247-5b88-4549-8e3f-afe9b73485fc None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:05.464391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffffd7-2953-42e8-8daf-45784c83c430 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:05.502492 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e6e92-2bb9-48c5-ae4f-01a23d7ebf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:05.503261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e6e92-2bb9-48c5-ae4f-01a23d7ebf5a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:05.503539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e6e92-2bb9-48c5-ae4f-01a23d7ebf5a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:05.503907 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e6e92-2bb9-48c5-ae4f-01a23d7ebf5a None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:05.504552 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e6e92-2bb9-48c5-ae4f-01a23d7ebf5a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:05.504852 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e6e92-2bb9-48c5-ae4f-01a23d7ebf5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:05.506239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e6e92-2bb9-48c5-ae4f-01a23d7ebf5a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:05.512614 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 375/756] 10.222.0.78 () {56 vars in 1124 bytes} [Mon Sep 19 11:06:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:05.513650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e6e92-2bb9-48c5-ae4f-01a23d7ebf5a None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:05.529704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:05.537792 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ff247-5b88-4549-8e3f-afe9b73485fc None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:05.563367 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:05.577011 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d1223-6c72-4093-84f3-93b034bb2269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:05.577696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d1223-6c72-4093-84f3-93b034bb2269 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:05.577949 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d1223-6c72-4093-84f3-93b034bb2269 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:05.582865 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d1223-6c72-4093-84f3-93b034bb2269 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:05.588060 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d1223-6c72-4093-84f3-93b034bb2269 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:05.589369 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d1223-6c72-4093-84f3-93b034bb2269 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:05.609451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-b32d1223-6c72-4093-84f3-93b034bb2269 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:06:05.609451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:05.609451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:05.609451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:05.609451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:05.609451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:05.609451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:05.609451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:05.609451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:05.609451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:05.609451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:05.609451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:05.609451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:06:05.609451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:06:05.609451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:06:05.609451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:05.610644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:05.611683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:05.611683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:05.611683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:06:05.611683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:06:05.611683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:06:05.611683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:06:05.611683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:06:05.611683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:06:05.611683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:06:05.611683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:05.611683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 382/757] 10.222.0.78 () {56 vars in 1093 bytes} [Mon Sep 19 11:06:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:05.612708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d1223-6c72-4093-84f3-93b034bb2269 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:05.632982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e6e92-2bb9-48c5-ae4f-01a23d7ebf5a None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:05.670058 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0be67-f24c-49f9-a688-69282b17c572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:05.671050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0be67-f24c-49f9-a688-69282b17c572 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:05.671515 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0be67-f24c-49f9-a688-69282b17c572 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:05.671962 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0be67-f24c-49f9-a688-69282b17c572 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:05.681250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0be67-f24c-49f9-a688-69282b17c572 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:05.683114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0be67-f24c-49f9-a688-69282b17c572 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:05.697733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-e4f0be67-f24c-49f9-a688-69282b17c572 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:06:05.697733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:05.697733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:05.697733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:05.697733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:05.697733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:05.697733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:05.697733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:05.697733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:05.697733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:05.697733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:05.697733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:05.697733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:06:05.697733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:06:05.697733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:06:05.697733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:05.698906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:05.699943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:05.699943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:05.699943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:06:05.699943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:06:05.699943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:06:05.699943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:06:05.699943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:06:05.699943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:06:05.699943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:06:05.699943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:05.702233 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 376/758] 10.222.0.78 () {56 vars in 1117 bytes} [Mon Sep 19 11:06:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:05.708127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0be67-f24c-49f9-a688-69282b17c572 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:05.748037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d1223-6c72-4093-84f3-93b034bb2269 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:05.783995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:05.791267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6ae9d-0bd5-4ade-a96d-62a54a3752bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:05.794417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6ae9d-0bd5-4ade-a96d-62a54a3752bf None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:05.794417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6ae9d-0bd5-4ade-a96d-62a54a3752bf None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:05.794417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6ae9d-0bd5-4ade-a96d-62a54a3752bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:05.794417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6ae9d-0bd5-4ade-a96d-62a54a3752bf None admin] RBAC: Adding query filter params (interface=public, service_id=0fa7163469964f19a892ff2e773e7ed1, region_id=RegionOne) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:05.794417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6ae9d-0bd5-4ade-a96d-62a54a3752bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:05.795621 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6ae9d-0bd5-4ade-a96d-62a54a3752bf None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:05.805709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 383/759] 10.222.0.78 () {56 vars in 1251 bytes} [Mon Sep 19 11:06:05 2022] GET /identity/v3/endpoints?service_id=0fa7163469964f19a892ff2e773e7ed1&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:05.806800 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6ae9d-0bd5-4ade-a96d-62a54a3752bf None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:05.819394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0be67-f24c-49f9-a688-69282b17c572 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:05.856769 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a888b94-fbfe-489a-b8a3-90dad3cabec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:05.857455 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a888b94-fbfe-489a-b8a3-90dad3cabec9 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:05.857798 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a888b94-fbfe-489a-b8a3-90dad3cabec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:05.859005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a888b94-fbfe-489a-b8a3-90dad3cabec9 None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:05.859005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a888b94-fbfe-489a-b8a3-90dad3cabec9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:05.859404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a888b94-fbfe-489a-b8a3-90dad3cabec9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:05.860897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a888b94-fbfe-489a-b8a3-90dad3cabec9 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:05.876076 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:05.888426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 377/760] 10.222.0.78 () {56 vars in 1133 bytes} [Mon Sep 19 11:06:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:05.888426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a888b94-fbfe-489a-b8a3-90dad3cabec9 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:05.955773 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6ae9d-0bd5-4ade-a96d-62a54a3752bf None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:06.002541 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf2bf7-761c-4309-a842-a0f13c49fd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:06.003086 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf2bf7-761c-4309-a842-a0f13c49fd0b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:06.003391 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf2bf7-761c-4309-a842-a0f13c49fd0b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:06.003661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf2bf7-761c-4309-a842-a0f13c49fd0b None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:06.004623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf2bf7-761c-4309-a842-a0f13c49fd0b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:06.004926 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf2bf7-761c-4309-a842-a0f13c49fd0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:06.006647 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf2bf7-761c-4309-a842-a0f13c49fd0b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:06.012552 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a888b94-fbfe-489a-b8a3-90dad3cabec9 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:06.033589 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 384/761] 10.222.0.78 () {56 vars in 1149 bytes} [Mon Sep 19 11:06:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:06.034660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf2bf7-761c-4309-a842-a0f13c49fd0b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:06.057660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:06.065766 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a3c78-1095-4989-bfdf-01e9ba555e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:06.066374 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a3c78-1095-4989-bfdf-01e9ba555e50 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:06.066650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a3c78-1095-4989-bfdf-01e9ba555e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:06.070633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a3c78-1095-4989-bfdf-01e9ba555e50 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:06.070633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a3c78-1095-4989-bfdf-01e9ba555e50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:06.070633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a3c78-1095-4989-bfdf-01e9ba555e50 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:06.075553 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 378/762] 10.222.0.78 () {56 vars in 1087 bytes} [Mon Sep 19 11:06:05 2022] GET /identity/v3/services => generated 772 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:06.076556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a3c78-1095-4989-bfdf-01e9ba555e50 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:06.163352 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf2bf7-761c-4309-a842-a0f13c49fd0b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:06.191200 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a3c78-1095-4989-bfdf-01e9ba555e50 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:06.232925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:06.237245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a49f7-c8ba-4e57-a82b-b57aa2b4add2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:06.237876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a49f7-c8ba-4e57-a82b-b57aa2b4add2 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:06.238152 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a49f7-c8ba-4e57-a82b-b57aa2b4add2 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:06.238420 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a49f7-c8ba-4e57-a82b-b57aa2b4add2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:06.303440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73182a0-8731-4bac-b841-a6c3e49b75cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:06.304078 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73182a0-8731-4bac-b841-a6c3e49b75cc None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:06.304993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73182a0-8731-4bac-b841-a6c3e49b75cc None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:06.305364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73182a0-8731-4bac-b841-a6c3e49b75cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:06.309857 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:06.318942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a49f7-c8ba-4e57-a82b-b57aa2b4add2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:06.320724 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a49f7-c8ba-4e57-a82b-b57aa2b4add2 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:06.331976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73182a0-8731-4bac-b841-a6c3e49b75cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:06.334075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73182a0-8731-4bac-b841-a6c3e49b75cc None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:06.361876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-a73182a0-8731-4bac-b841-a6c3e49b75cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:06.361876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:06.361876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:06.361876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:06.361876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:06.361876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:06.361876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:06.361876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:06.361876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:06.361876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:06.361876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:06.361876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:06.361876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:06.361876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:06.361876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:06.361876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:06.363099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:06.363099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:06.363099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:06.363099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:06.363099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:06.363099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:06.363099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:06.363099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:06.363099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:06.363099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:06.363099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:06.363099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:06.363099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:06.363099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:06.363099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:06.363099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:06.363099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:06.364044 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:06.364044 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:06.364044 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:06.364651 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 385/763] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:06.372459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73182a0-8731-4bac-b841-a6c3e49b75cc None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:06.440780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 379/764] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:06:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:06:06.446872 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a49f7-c8ba-4e57-a82b-b57aa2b4add2 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:06.526058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73182a0-8731-4bac-b841-a6c3e49b75cc None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:06.544533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:06.558654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a49f7-c8ba-4e57-a82b-b57aa2b4add2 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:06.572486 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8798e4-746c-4720-ba97-01d04e94046f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:06.573273 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8798e4-746c-4720-ba97-01d04e94046f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:06.573675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8798e4-746c-4720-ba97-01d04e94046f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:06.574051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8798e4-746c-4720-ba97-01d04e94046f None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:06.574780 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8798e4-746c-4720-ba97-01d04e94046f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:06.575146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8798e4-746c-4720-ba97-01d04e94046f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:06.576798 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8798e4-746c-4720-ba97-01d04e94046f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:06.591480 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 386/765] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:06.599024 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8798e4-746c-4720-ba97-01d04e94046f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:06.606314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e8021-20ee-4e38-9a19-dbc83dcdb87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:06.606955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e8021-20ee-4e38-9a19-dbc83dcdb87d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:06.607251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e8021-20ee-4e38-9a19-dbc83dcdb87d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:06.607522 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e8021-20ee-4e38-9a19-dbc83dcdb87d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:06.628451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e8021-20ee-4e38-9a19-dbc83dcdb87d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:06.629924 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e8021-20ee-4e38-9a19-dbc83dcdb87d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:06.643918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:06.661168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-036e8021-20ee-4e38-9a19-dbc83dcdb87d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:06.661168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:06.661168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:06.661168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:06.661168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:06.661168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:06.661168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:06.661168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:06.661168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:06.661168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:06.661168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:06.661168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:06.661168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:06.661168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:06.661168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:06.661168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:06.662402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:06.662402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:06.662402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:06.662402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:06.662402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:06.662402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:06.662402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:06.662402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:06.662402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:06.662402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:06.662402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:06.662402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:06.662402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:06.662402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:06.662402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:06.662402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:06.662402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:06.663568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:06.663568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:06.663568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:06.663568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 380/766] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:06.676296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e8021-20ee-4e38-9a19-dbc83dcdb87d None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:06.791877 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8798e4-746c-4720-ba97-01d04e94046f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:06.859378 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d4551-cceb-44e3-9487-a3acc2d05303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:06.859943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d4551-cceb-44e3-9487-a3acc2d05303 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:06.860223 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d4551-cceb-44e3-9487-a3acc2d05303 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:06.860495 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d4551-cceb-44e3-9487-a3acc2d05303 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:06.869745 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e8021-20ee-4e38-9a19-dbc83dcdb87d None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:06.873091 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d4551-cceb-44e3-9487-a3acc2d05303 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:06.878879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d4551-cceb-44e3-9487-a3acc2d05303 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:06.894187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-2a7d4551-cceb-44e3-9487-a3acc2d05303 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:06.894187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:06.894187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:06.894187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:06.894187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:06.894187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:06.894187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:06.894187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:06.894187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:06.894187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:06.894187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:06.894187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:06.894187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:06:06.894187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:06:06.894187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:06:06.895256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:06:06.895256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:06.895256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:06.895256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:06.895256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:06.895256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:06.895256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:06.895256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:06.895256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:06.895256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:06.895256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:06.895256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:06.895256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:06.895256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:06.895256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:06.895256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:06.895256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:06.896147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:06:06.896147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:06:06.896147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:06:06.896147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:06:06.896147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:06:06.896147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:06:06.896147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:06.896147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:06.896147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 387/767] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:06.903775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d4551-cceb-44e3-9487-a3acc2d05303 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:06.943422 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa83da3-5d52-483a-85f0-8a24aa58af33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:06.944048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa83da3-5d52-483a-85f0-8a24aa58af33 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:06.944326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa83da3-5d52-483a-85f0-8a24aa58af33 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:06.944597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa83da3-5d52-483a-85f0-8a24aa58af33 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:06.945793 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa83da3-5d52-483a-85f0-8a24aa58af33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:06.946095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa83da3-5d52-483a-85f0-8a24aa58af33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:06.947688 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa83da3-5d52-483a-85f0-8a24aa58af33 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:06.978039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 381/768] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:06.983731 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa83da3-5d52-483a-85f0-8a24aa58af33 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:06.994421 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:07.084192 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:07.116687 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d4551-cceb-44e3-9487-a3acc2d05303 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:07.168319 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa83da3-5d52-483a-85f0-8a24aa58af33 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:07.180462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10feca0a-b009-44e3-8374-df9630435242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:07.181578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10feca0a-b009-44e3-8374-df9630435242 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:07.181986 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10feca0a-b009-44e3-8374-df9630435242 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:07.182511 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10feca0a-b009-44e3-8374-df9630435242 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:07.183442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10feca0a-b009-44e3-8374-df9630435242 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:07.188083 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10feca0a-b009-44e3-8374-df9630435242 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:07.188083 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10feca0a-b009-44e3-8374-df9630435242 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:07.203847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 388/769] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:06:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:07.210346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf0928-82bf-4455-b4cb-484987351aad None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:07.210936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf0928-82bf-4455-b4cb-484987351aad None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:07.211443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf0928-82bf-4455-b4cb-484987351aad None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:07.213225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 389/770] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:07.214146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf0928-82bf-4455-b4cb-484987351aad None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:07.221457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41ee13-eb91-4512-aafe-b86b39217ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:07.222109 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41ee13-eb91-4512-aafe-b86b39217ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:07.222390 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41ee13-eb91-4512-aafe-b86b39217ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:07.222666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41ee13-eb91-4512-aafe-b86b39217ce7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:07.232988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41ee13-eb91-4512-aafe-b86b39217ce7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:07.234449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41ee13-eb91-4512-aafe-b86b39217ce7 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:07.251849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-9f41ee13-eb91-4512-aafe-b86b39217ce7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:07.251849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:07.251849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:07.251849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:07.251849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:07.251849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:07.251849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:07.251849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:07.251849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:07.251849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:07.251849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:07.251849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:07.251849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:06:07.251849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:06:07.251849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:06:07.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:06:07.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:07.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:07.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:07.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:07.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:07.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:07.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:07.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:07.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:07.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:07.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:07.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:07.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:07.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:07.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:07.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:07.253700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:06:07.253700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:06:07.253700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:06:07.253700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:06:07.253700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:06:07.253700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:06:07.253700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:07.253700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:07.261027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 382/771] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:07.261027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405e724-9c6a-4388-bcae-fb8195ea7c06 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:07.261027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405e724-9c6a-4388-bcae-fb8195ea7c06 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:07.261027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405e724-9c6a-4388-bcae-fb8195ea7c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:07.276693 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-e405e724-9c6a-4388-bcae-fb8195ea7c06 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:07.277898 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e405e724-9c6a-4388-bcae-fb8195ea7c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:07.342696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf0928-82bf-4455-b4cb-484987351aad None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:07.354081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:07.367630 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 383/772] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:07.369103 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d14b6-c160-4f93-a7cf-bf3ddbac9059 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:07.369263 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d14b6-c160-4f93-a7cf-bf3ddbac9059 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:07.369493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d14b6-c160-4f93-a7cf-bf3ddbac9059 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:07.370769 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 384/773] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:07.374720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d14b6-c160-4f93-a7cf-bf3ddbac9059 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:07.387638 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4f1ae-6dd7-4bc4-8904-4983f7550c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:07.388351 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4f1ae-6dd7-4bc4-8904-4983f7550c05 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:07.388627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4f1ae-6dd7-4bc4-8904-4983f7550c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:07.388899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4f1ae-6dd7-4bc4-8904-4983f7550c05 None admin] PATH_INFO: `/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/aba1968529574fe5b4c5d114c874f428/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:07.392524 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4f1ae-6dd7-4bc4-8904-4983f7550c05 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8563e86b7c94ff29e39fb20a7969551, user_id=aba1968529574fe5b4c5d114c874f428, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:07.394795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4f1ae-6dd7-4bc4-8904-4983f7550c05 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:07.463799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:07.475657 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 390/774] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:06:07 2022] PUT /identity/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/aba1968529574fe5b4c5d114c874f428/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:06:07.478299 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5f718-5cab-4027-adea-d82766f33ddf None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:07.478585 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5f718-5cab-4027-adea-d82766f33ddf None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:07.478869 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5f718-5cab-4027-adea-d82766f33ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:07.518999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0ae5f718-5cab-4027-adea-d82766f33ddf None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:07.521794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0ae5f718-5cab-4027-adea-d82766f33ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:07.534809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d14b6-c160-4f93-a7cf-bf3ddbac9059 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:07.605707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8beb3-f148-47df-adf9-22ada4d94f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:07.606392 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8beb3-f148-47df-adf9-22ada4d94f2a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:07.606669 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8beb3-f148-47df-adf9-22ada4d94f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:07.606946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8beb3-f148-47df-adf9-22ada4d94f2a None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:07.607663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8beb3-f148-47df-adf9-22ada4d94f2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:07.607966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8beb3-f148-47df-adf9-22ada4d94f2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:07.609427 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8beb3-f148-47df-adf9-22ada4d94f2a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:07.637905 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 385/775] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:06:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:07.640315 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8beb3-f148-47df-adf9-22ada4d94f2a None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:07.662098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 391/776] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:07.662988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5f718-5cab-4027-adea-d82766f33ddf None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:07.731095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:07.779064 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8beb3-f148-47df-adf9-22ada4d94f2a None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:07.782567 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5f718-5cab-4027-adea-d82766f33ddf None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:07.812543 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:07.829660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ee8e4-4b27-446e-81d2-bb49a5f4d559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:07.830600 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ee8e4-4b27-446e-81d2-bb49a5f4d559 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:07.831080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ee8e4-4b27-446e-81d2-bb49a5f4d559 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:07.831550 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ee8e4-4b27-446e-81d2-bb49a5f4d559 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:07.835269 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd024127-f748-4330-b037-a8325c6c16ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:07.835269 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd024127-f748-4330-b037-a8325c6c16ce None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:07.835269 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd024127-f748-4330-b037-a8325c6c16ce None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:07.835269 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd024127-f748-4330-b037-a8325c6c16ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:07.836369 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd024127-f748-4330-b037-a8325c6c16ce None admin] RBAC: Adding query filter params (role.id=28365abffd3d4ff6874bce2e0c561f1d, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=5a667fa07b1148998959835d9995ad60) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:07.837998 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd024127-f748-4330-b037-a8325c6c16ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:07.838680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd024127-f748-4330-b037-a8325c6c16ce None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:07.849806 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 386/777] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:06:07 2022] GET /identity/v3/role_assignments?user.id=5a667fa07b1148998959835d9995ad60&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=28365abffd3d4ff6874bce2e0c561f1d => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:07.861170 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ee8e4-4b27-446e-81d2-bb49a5f4d559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:06:07.872619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-667ee8e4-4b27-446e-81d2-bb49a5f4d559 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:06:07.872619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:07.872619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:07.872619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:07.872619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:07.872619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:07.872619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:07.872619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:07.872619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:07.872619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:07.872619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:07.872619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:07.872619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:06:07.872619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:06:07.872619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:06:07.872619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:06:07.873837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:06:07.873837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:06:07.873837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:06:07.873837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:06:07.873837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:07.873837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:07.873837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:07.873837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:07.873837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:07.873837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:07.873837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:07.873837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:07.873837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:07.873837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:07.873837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:07.873837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:07.874901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:07.874901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:07.874901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:07.874901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:07.874901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:06:07.874901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:06:07.874901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:06:07.874901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:06:07.874901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:06:07.874901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:06:07.874901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:06:07.874901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:07.877423 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 392/778] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:06:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:07.884228 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd024127-f748-4330-b037-a8325c6c16ce None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:07.982809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ee8e4-4b27-446e-81d2-bb49a5f4d559 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:08.021942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd024127-f748-4330-b037-a8325c6c16ce None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:08.047757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:08.061019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab5229-febb-41d9-9eba-e58083f59c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:08.061653 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab5229-febb-41d9-9eba-e58083f59c4a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:08.061932 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab5229-febb-41d9-9eba-e58083f59c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:08.063603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab5229-febb-41d9-9eba-e58083f59c4a None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:08.063603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab5229-febb-41d9-9eba-e58083f59c4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:08.063603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab5229-febb-41d9-9eba-e58083f59c4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:08.064899 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab5229-febb-41d9-9eba-e58083f59c4a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:08.075850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 387/779] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:06:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:08.082896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab5229-febb-41d9-9eba-e58083f59c4a None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:08.145467 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ee8e4-4b27-446e-81d2-bb49a5f4d559 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:08.212655 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcf4a3-dfa3-4ae3-b230-43b725987eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:08.213703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcf4a3-dfa3-4ae3-b230-43b725987eed None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:08.215228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcf4a3-dfa3-4ae3-b230-43b725987eed None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:08.215734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcf4a3-dfa3-4ae3-b230-43b725987eed None admin] PATH_INFO: `/v3/services/network` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:08.216706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcf4a3-dfa3-4ae3-b230-43b725987eed None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:08.219665 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcf4a3-dfa3-4ae3-b230-43b725987eed None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:08.238705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-0efcf4a3-dfa3-4ae3-b230-43b725987eed None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:06:08.238705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:08.238705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:08.238705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:08.238705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:08.238705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:08.238705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:08.238705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:08.238705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:06:08.238705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:08.238705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:08.238705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:08.238705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:08.238705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:08.238705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:08.238705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:08.238705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:08.239789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:08.239789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:08.239789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:08.239789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:08.239789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:08.239789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:08.239789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:08.239789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:08.239789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:08.239789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:08.239789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:08.239789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:08.239789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:08.239789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:08.239789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:08.239789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:08.240635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:08.240635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:08.240635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:08.240635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:08.240635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:08.240635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:08.240635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:08.240635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:08.240635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:08.240635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:08.240635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:08.240635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:08.240635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:06:08.240635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:08.244485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab5229-febb-41d9-9eba-e58083f59c4a None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:08.248640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 393/780] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:07 2022] GET /identity/v3/services/network => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:08.254654 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcf4a3-dfa3-4ae3-b230-43b725987eed None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:08.310202 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539cfbd-d146-4534-b572-6a048448c04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:08.310850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539cfbd-d146-4534-b572-6a048448c04b None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:08.311129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539cfbd-d146-4534-b572-6a048448c04b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:08.311424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539cfbd-d146-4534-b572-6a048448c04b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:08.334078 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539cfbd-d146-4534-b572-6a048448c04b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:08.335595 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539cfbd-d146-4534-b572-6a048448c04b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:08.354669 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:08.359486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-9539cfbd-d146-4534-b572-6a048448c04b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:08.359486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:08.359486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:08.359486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:08.359486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:08.359486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:08.359486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:08.359486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:08.359486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:08.359486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:08.359486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:08.359486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:08.359486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:08.359486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:08.359486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:08.359486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:08.360467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:08.360467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:08.360467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:08.360467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:08.360467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:08.360467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:08.360467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:08.360467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:08.360467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:08.360467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:08.360467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:08.360467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:08.360467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:08.360467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:08.360467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:08.360467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:08.360467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:08.361509 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:08.361509 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:08.361509 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:08.362068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 388/781] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:08.372233 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539cfbd-d146-4534-b572-6a048448c04b None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:08.419564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:08.444786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcf4a3-dfa3-4ae3-b230-43b725987eed None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:08.481698 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6305c-84c7-4397-9214-087774be5967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:08.482691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6305c-84c7-4397-9214-087774be5967 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:08.483238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6305c-84c7-4397-9214-087774be5967 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:08.483718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6305c-84c7-4397-9214-087774be5967 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:08.484559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6305c-84c7-4397-9214-087774be5967 None admin] RBAC: Adding query filter params (name=network) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:08.485068 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6305c-84c7-4397-9214-087774be5967 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:08.486635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6305c-84c7-4397-9214-087774be5967 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:08.492844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 394/782] 10.222.0.78 () {56 vars in 1112 bytes} [Mon Sep 19 11:06:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:08.502453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6305c-84c7-4397-9214-087774be5967 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:08.528280 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539cfbd-d146-4534-b572-6a048448c04b None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:08.567983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86947888-6d61-4d55-b7e9-c5de8208e241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:08.568598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86947888-6d61-4d55-b7e9-c5de8208e241 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:08.568872 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86947888-6d61-4d55-b7e9-c5de8208e241 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:08.569140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86947888-6d61-4d55-b7e9-c5de8208e241 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:08.569813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86947888-6d61-4d55-b7e9-c5de8208e241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:08.570119 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86947888-6d61-4d55-b7e9-c5de8208e241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:08.571599 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86947888-6d61-4d55-b7e9-c5de8208e241 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:08.584816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 389/783] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:08.591211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86947888-6d61-4d55-b7e9-c5de8208e241 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:08.696143 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6305c-84c7-4397-9214-087774be5967 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:08.714646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:08.733446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98041f85-5981-4375-ba3c-ea698845df02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:08.734071 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98041f85-5981-4375-ba3c-ea698845df02 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:08.734372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98041f85-5981-4375-ba3c-ea698845df02 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:08.734911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98041f85-5981-4375-ba3c-ea698845df02 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:08.735275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98041f85-5981-4375-ba3c-ea698845df02 None admin] RBAC: Adding query filter params (type=network) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:08.735579 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98041f85-5981-4375-ba3c-ea698845df02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:08.736944 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98041f85-5981-4375-ba3c-ea698845df02 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:08.741969 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:08.742921 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 395/784] 10.222.0.78 () {56 vars in 1112 bytes} [Mon Sep 19 11:06:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:08.747239 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcc2ef-3f58-4f09-bf32-a3f994d2011a None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:08.747239 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcc2ef-3f58-4f09-bf32-a3f994d2011a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:08.747239 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcc2ef-3f58-4f09-bf32-a3f994d2011a None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:08.750328 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 396/785] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:08.751544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcc2ef-3f58-4f09-bf32-a3f994d2011a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:08.758446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86947888-6d61-4d55-b7e9-c5de8208e241 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:08.810160 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e0c8e-9a8b-40b8-87f6-9a11b327b4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:08.810853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e0c8e-9a8b-40b8-87f6-9a11b327b4ed None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:08.811136 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e0c8e-9a8b-40b8-87f6-9a11b327b4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:08.811437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e0c8e-9a8b-40b8-87f6-9a11b327b4ed None admin] PATH_INFO: `/v3/users/swift` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:08.819568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e0c8e-9a8b-40b8-87f6-9a11b327b4ed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:08.821059 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e0c8e-9a8b-40b8-87f6-9a11b327b4ed None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:08.834335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-3e6e0c8e-9a8b-40b8-87f6-9a11b327b4ed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:06:08.834335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:08.834335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:08.834335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:08.834335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:08.834335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:08.834335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:08.834335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:08.834335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:08.834335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:08.834335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:08.834335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:08.834335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:06:08.834335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:06:08.834335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:06:08.834335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:08.835341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:08.836835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:08.836835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:08.836835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:06:08.836835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:06:08.836835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:06:08.836835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:06:08.836835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:06:08.836835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:06:08.836835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:06:08.836835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:08.836835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 390/786] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:06:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:08.838064 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd9924-cb98-4bd6-8ccb-f08a221f2fd0 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:08.838223 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd9924-cb98-4bd6-8ccb-f08a221f2fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:08.838375 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd9924-cb98-4bd6-8ccb-f08a221f2fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:08.859061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-bdfd9924-cb98-4bd6-8ccb-f08a221f2fd0 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:08.860421 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-bdfd9924-cb98-4bd6-8ccb-f08a221f2fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:08.866100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcc2ef-3f58-4f09-bf32-a3f994d2011a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:08.900936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab9210-d7c0-4166-980f-7735188ec1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:08.901602 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab9210-d7c0-4166-980f-7735188ec1af None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:08.901936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab9210-d7c0-4166-980f-7735188ec1af None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:08.902211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab9210-d7c0-4166-980f-7735188ec1af None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:08.902914 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab9210-d7c0-4166-980f-7735188ec1af None admin] RBAC: Adding query filter params (interface=public, service_id=8049e0e2675e42f29126c5e127edc9c4, region_id=RegionOne) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:08.903240 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab9210-d7c0-4166-980f-7735188ec1af None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:08.904608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab9210-d7c0-4166-980f-7735188ec1af None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:08.915159 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 397/787] 10.222.0.78 () {56 vars in 1251 bytes} [Mon Sep 19 11:06:08 2022] GET /identity/v3/endpoints?service_id=8049e0e2675e42f29126c5e127edc9c4&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:08.916110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab9210-d7c0-4166-980f-7735188ec1af None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:08.943413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 391/788] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:08.944573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd9924-cb98-4bd6-8ccb-f08a221f2fd0 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:08.963288 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:09.042988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab9210-d7c0-4166-980f-7735188ec1af None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:09.046172 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:09.064094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd9924-cb98-4bd6-8ccb-f08a221f2fd0 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:09.078890 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb35877-079a-4b9d-a3d2-520fa192d128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:09.079768 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb35877-079a-4b9d-a3d2-520fa192d128 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:09.080230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb35877-079a-4b9d-a3d2-520fa192d128 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:09.080658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb35877-079a-4b9d-a3d2-520fa192d128 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:09.081524 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb35877-079a-4b9d-a3d2-520fa192d128 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:09.081971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb35877-079a-4b9d-a3d2-520fa192d128 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:09.083564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb35877-079a-4b9d-a3d2-520fa192d128 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:09.103798 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbc641-cb27-456f-9cae-6c1973a3fd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:09.104423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbc641-cb27-456f-9cae-6c1973a3fd25 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:09.104704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbc641-cb27-456f-9cae-6c1973a3fd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:09.104978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbc641-cb27-456f-9cae-6c1973a3fd25 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:09.105585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbc641-cb27-456f-9cae-6c1973a3fd25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:09.106969 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbc641-cb27-456f-9cae-6c1973a3fd25 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:09.113753 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 392/789] 10.222.0.78 () {56 vars in 1087 bytes} [Mon Sep 19 11:06:08 2022] GET /identity/v3/services => generated 772 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:09.125620 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 398/790] 10.222.0.78 () {56 vars in 1135 bytes} [Mon Sep 19 11:06:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:09.131527 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbc641-cb27-456f-9cae-6c1973a3fd25 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:09.249889 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbc641-cb27-456f-9cae-6c1973a3fd25 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:09.290781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a7c57-ae3b-4a94-b242-558d577bd553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:09.291462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a7c57-ae3b-4a94-b242-558d577bd553 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:09.291744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a7c57-ae3b-4a94-b242-558d577bd553 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:09.292018 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a7c57-ae3b-4a94-b242-558d577bd553 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:09.304598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a7c57-ae3b-4a94-b242-558d577bd553 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:09.305990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a7c57-ae3b-4a94-b242-558d577bd553 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:09.323900 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb35877-079a-4b9d-a3d2-520fa192d128 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:09.336443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:09.349700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-578a7c57-ae3b-4a94-b242-558d577bd553 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:09.349700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:09.349700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:09.349700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:09.349700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:09.349700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:09.349700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:09.349700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:09.349700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:09.349700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:09.349700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:09.349700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:09.349700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:09.349700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:09.349700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:09.349700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:09.350687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:09.350687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:09.350687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:09.350687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:09.350687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:09.350687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:09.350687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:09.350687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:09.350687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:09.350687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:09.350687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:09.350687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:09.350687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:09.350687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:09.350687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:09.350687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:09.350687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:09.351652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:09.351652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:09.351652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:09.356280 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 393/791] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:09.364418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a7c57-ae3b-4a94-b242-558d577bd553 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:09.592587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb35877-079a-4b9d-a3d2-520fa192d128 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:09.595916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:09.634684 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40af0a-a79e-4208-bd09-3158ba807275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:09.638500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40af0a-a79e-4208-bd09-3158ba807275 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:09.639112 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40af0a-a79e-4208-bd09-3158ba807275 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:09.639796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40af0a-a79e-4208-bd09-3158ba807275 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:09.640642 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40af0a-a79e-4208-bd09-3158ba807275 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:09.642946 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40af0a-a79e-4208-bd09-3158ba807275 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:09.668537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-2d40af0a-a79e-4208-bd09-3158ba807275 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:06:09.668537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:09.668537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:09.668537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:09.668537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:09.668537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:09.668537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:09.668537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:09.668537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:06:09.668537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:09.668537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:09.668537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:09.668537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:09.668537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:09.668537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:09.668537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:09.668537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:09.669599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:09.669599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:09.669599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:09.669599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:09.669599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:09.669599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:09.669599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:09.669599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:09.669599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:09.669599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:09.669599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:09.669599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:09.669599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:09.669599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:09.669599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:09.669599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:09.670453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:09.670453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:09.670453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:09.670453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:09.670453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:09.670453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:09.670453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:09.670453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:09.670453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:09.670453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:09.670453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:09.670453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:09.670453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:06:09.670453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:09.674258 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 399/792] 10.222.0.78 () {56 vars in 1129 bytes} [Mon Sep 19 11:06:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:09.682942 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40af0a-a79e-4208-bd09-3158ba807275 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:09.797531 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a7c57-ae3b-4a94-b242-558d577bd553 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:09.906757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e8a7b-4f7b-4582-817f-4339c2be1f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:09.907959 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e8a7b-4f7b-4582-817f-4339c2be1f72 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:09.908807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e8a7b-4f7b-4582-817f-4339c2be1f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:09.909441 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e8a7b-4f7b-4582-817f-4339c2be1f72 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:09.912529 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e8a7b-4f7b-4582-817f-4339c2be1f72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:09.912529 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e8a7b-4f7b-4582-817f-4339c2be1f72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:09.914472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e8a7b-4f7b-4582-817f-4339c2be1f72 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:09.946334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 394/793] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 582 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:09.951485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e8a7b-4f7b-4582-817f-4339c2be1f72 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:09.963071 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40af0a-a79e-4208-bd09-3158ba807275 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:10.037450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24b741-c217-4f44-8ece-7985ce681720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:10.039817 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24b741-c217-4f44-8ece-7985ce681720 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:10.040344 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24b741-c217-4f44-8ece-7985ce681720 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:10.040821 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24b741-c217-4f44-8ece-7985ce681720 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:10.041669 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24b741-c217-4f44-8ece-7985ce681720 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:10.042171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24b741-c217-4f44-8ece-7985ce681720 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:10.043893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24b741-c217-4f44-8ece-7985ce681720 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:10.049623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 400/794] 10.222.0.78 () {56 vars in 1124 bytes} [Mon Sep 19 11:06:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:10.056333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24b741-c217-4f44-8ece-7985ce681720 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:10.091542 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:10.129652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:10.179889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24b741-c217-4f44-8ece-7985ce681720 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:10.219068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e8a7b-4f7b-4582-817f-4339c2be1f72 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:10.221039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fd467-3801-498b-9836-95bd8c08c6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:10.222026 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fd467-3801-498b-9836-95bd8c08c6ee None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:10.222525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fd467-3801-498b-9836-95bd8c08c6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:10.223023 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fd467-3801-498b-9836-95bd8c08c6ee None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:10.223956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30fd467-3801-498b-9836-95bd8c08c6ee None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:10.224470 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30fd467-3801-498b-9836-95bd8c08c6ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:10.226639 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30fd467-3801-498b-9836-95bd8c08c6ee None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:10.232778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 401/795] 10.222.0.78 () {56 vars in 1124 bytes} [Mon Sep 19 11:06:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:10.238786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fd467-3801-498b-9836-95bd8c08c6ee None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:10.282642 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7b6f9-01be-490d-8d2e-d2eb9953b3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:10.283312 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7b6f9-01be-490d-8d2e-d2eb9953b3fc None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:10.283691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7b6f9-01be-490d-8d2e-d2eb9953b3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:10.283976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7b6f9-01be-490d-8d2e-d2eb9953b3fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:10.299267 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7b6f9-01be-490d-8d2e-d2eb9953b3fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:10.303068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7b6f9-01be-490d-8d2e-d2eb9953b3fc None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:10.317402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-ffb7b6f9-01be-490d-8d2e-d2eb9953b3fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:10.317402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:10.317402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:10.317402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:10.317402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:10.317402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:10.317402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:10.317402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:10.317402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:10.317402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:10.317402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:10.317402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:10.317402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:06:10.317402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:06:10.317402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:06:10.318339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:06:10.318339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:10.318339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:10.318339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:10.318339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:10.318339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:10.318339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:10.318339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:10.318339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:10.318339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:10.318339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:10.318339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:10.318339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:10.318339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:10.318339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:10.318339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:10.318339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:10.319270 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:06:10.319270 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:06:10.319270 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:06:10.319270 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:06:10.319270 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:06:10.319270 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:06:10.319270 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:10.319270 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:10.322313 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 395/796] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:10.324620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134bd25-ac2e-4f84-85b3-78d8dacaa36b None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:10.324923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134bd25-ac2e-4f84-85b3-78d8dacaa36b None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:10.325179 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134bd25-ac2e-4f84-85b3-78d8dacaa36b None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:10.326714 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 396/797] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:06:10.331490 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134bd25-ac2e-4f84-85b3-78d8dacaa36b None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:10.366731 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:10.374068 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fd467-3801-498b-9836-95bd8c08c6ee None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:10.425154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2d482-d874-4915-9da0-085d4e62f238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:10.426298 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2d482-d874-4915-9da0-085d4e62f238 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:10.426793 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2d482-d874-4915-9da0-085d4e62f238 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:10.427305 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2d482-d874-4915-9da0-085d4e62f238 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:10.428209 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2d482-d874-4915-9da0-085d4e62f238 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:10.430052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2d482-d874-4915-9da0-085d4e62f238 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:10.442494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134bd25-ac2e-4f84-85b3-78d8dacaa36b None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:10.464354 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:10.479133 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408537a6-c156-46b9-9824-92f420038044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:10.479762 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408537a6-c156-46b9-9824-92f420038044 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:10.480043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408537a6-c156-46b9-9824-92f420038044 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:10.480311 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408537a6-c156-46b9-9824-92f420038044 None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:10.480995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408537a6-c156-46b9-9824-92f420038044 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:10.481300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408537a6-c156-46b9-9824-92f420038044 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:10.482724 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408537a6-c156-46b9-9824-92f420038044 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:10.494666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 397/798] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:06:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:10.499683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99f36a-1aef-4608-947b-50f2beb5e98a None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:10.499683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99f36a-1aef-4608-947b-50f2beb5e98a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:10.499683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99f36a-1aef-4608-947b-50f2beb5e98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:10.516824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-db99f36a-1aef-4608-947b-50f2beb5e98a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:10.518166 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-db99f36a-1aef-4608-947b-50f2beb5e98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:10.559156 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 402/799] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:06:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:06:10.561243 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2d482-d874-4915-9da0-085d4e62f238 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:10.652203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 398/800] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:10.653614 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e09871-7154-4dac-9740-312a9de9c42d None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:10.653909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e09871-7154-4dac-9740-312a9de9c42d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:10.654292 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e09871-7154-4dac-9740-312a9de9c42d None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:10.660001 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 399/801] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:10.661389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52baed6b-63f6-48c4-ac1c-2177593124eb None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:10.661676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52baed6b-63f6-48c4-ac1c-2177593124eb None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:10.661952 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52baed6b-63f6-48c4-ac1c-2177593124eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:10.681582 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-52baed6b-63f6-48c4-ac1c-2177593124eb None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:10.682452 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-52baed6b-63f6-48c4-ac1c-2177593124eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:10.727294 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2d482-d874-4915-9da0-085d4e62f238 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:10.766829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25728507-2b0f-42ce-b915-412096427545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:10.767889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25728507-2b0f-42ce-b915-412096427545 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:10.768372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25728507-2b0f-42ce-b915-412096427545 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:10.768903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25728507-2b0f-42ce-b915-412096427545 None admin] PATH_INFO: `/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/cf462b2aa3fc4e2ca58851549cf0c644/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:10.770820 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25728507-2b0f-42ce-b915-412096427545 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8563e86b7c94ff29e39fb20a7969551, user_id=cf462b2aa3fc4e2ca58851549cf0c644, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:10.772361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25728507-2b0f-42ce-b915-412096427545 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:10.817037 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 400/802] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:06:10.850054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 403/803] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:06:10 2022] PUT /identity/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/cf462b2aa3fc4e2ca58851549cf0c644/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:06:11.012893 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52baed6b-63f6-48c4-ac1c-2177593124eb None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:11.056961 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:11.131097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25728507-2b0f-42ce-b915-412096427545 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:11.151014 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:11.155610 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52baed6b-63f6-48c4-ac1c-2177593124eb None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:11.233957 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a93467-b669-4db6-94ca-1d0721ee5def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:11.234614 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a93467-b669-4db6-94ca-1d0721ee5def None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:11.234892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a93467-b669-4db6-94ca-1d0721ee5def None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:11.235183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a93467-b669-4db6-94ca-1d0721ee5def None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:11.256448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a93467-b669-4db6-94ca-1d0721ee5def None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:06:11.262101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-75a93467-b669-4db6-94ca-1d0721ee5def None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:06:11.262101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:11.262101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:11.262101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:11.262101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:11.262101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:11.262101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:11.262101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:11.262101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:11.262101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:11.262101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:11.262101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:11.262101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:06:11.262101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:06:11.262101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:06:11.262101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:06:11.263083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:06:11.263083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:06:11.263083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:06:11.263083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:06:11.263083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:11.263083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:11.263083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:11.263083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:11.263083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:11.263083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:11.263083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:11.263083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:11.263083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:11.263083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:11.263083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:11.263083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:11.264350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:11.264350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:11.264350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:11.264350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:11.264350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:06:11.264350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:06:11.264350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:06:11.264350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:06:11.264350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:06:11.264350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:06:11.264350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:06:11.264350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:11.264350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 401/804] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:06:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:11.274132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a93467-b669-4db6-94ca-1d0721ee5def None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:11.315670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25728507-2b0f-42ce-b915-412096427545 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:11.384724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78752132-36fc-4fe4-b372-d9b0c1f1c2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:11.385500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78752132-36fc-4fe4-b372-d9b0c1f1c2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:11.385783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78752132-36fc-4fe4-b372-d9b0c1f1c2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:11.386056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78752132-36fc-4fe4-b372-d9b0c1f1c2d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:11.411973 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78752132-36fc-4fe4-b372-d9b0c1f1c2d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:06:11.418913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-78752132-36fc-4fe4-b372-d9b0c1f1c2d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:06:11.418913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:11.418913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:11.418913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:11.418913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:11.418913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:11.418913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:11.418913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:11.418913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:11.418913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:11.418913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:11.418913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:11.418913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:06:11.418913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:06:11.418913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:06:11.418913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:06:11.419955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:06:11.419955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:06:11.419955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:06:11.419955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:06:11.419955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:11.419955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:11.419955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:11.419955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:11.419955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:11.419955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:11.419955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:11.419955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:11.419955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:11.419955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:11.419955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:11.419955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:11.420790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:11.420790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:11.420790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:11.420790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:11.420790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:06:11.420790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:06:11.420790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:06:11.420790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:06:11.420790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:06:11.420790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:06:11.420790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:06:11.420790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:11.421988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 404/805] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:06:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:11.430190 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78752132-36fc-4fe4-b372-d9b0c1f1c2d0 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:11.444539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:11.452221 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a93467-b669-4db6-94ca-1d0721ee5def None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:11.518325 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:11.530068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054911d-defc-4f1d-8fc7-a90fd29fd2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:11.532460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054911d-defc-4f1d-8fc7-a90fd29fd2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:11.532927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054911d-defc-4f1d-8fc7-a90fd29fd2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:11.533817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054911d-defc-4f1d-8fc7-a90fd29fd2a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:11.535146 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054911d-defc-4f1d-8fc7-a90fd29fd2a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:11.536429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054911d-defc-4f1d-8fc7-a90fd29fd2a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:11.537989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054911d-defc-4f1d-8fc7-a90fd29fd2a0 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:11.557513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 402/806] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:06:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:11.564687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054911d-defc-4f1d-8fc7-a90fd29fd2a0 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:11.573574 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78752132-36fc-4fe4-b372-d9b0c1f1c2d0 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:11.640524 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f319b8-3e7d-4591-90e0-17a04edb4cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:11.641163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f319b8-3e7d-4591-90e0-17a04edb4cda None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:11.641446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f319b8-3e7d-4591-90e0-17a04edb4cda None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:11.641721 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f319b8-3e7d-4591-90e0-17a04edb4cda None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:11.642551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f319b8-3e7d-4591-90e0-17a04edb4cda None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:11.642710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f319b8-3e7d-4591-90e0-17a04edb4cda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:11.644164 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f319b8-3e7d-4591-90e0-17a04edb4cda None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:11.658394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 405/807] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:06:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:11.664148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f319b8-3e7d-4591-90e0-17a04edb4cda None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:11.680741 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054911d-defc-4f1d-8fc7-a90fd29fd2a0 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:11.719867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c2e32-65fb-4da1-96b6-5a7fafaaeb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:11.720831 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c2e32-65fb-4da1-96b6-5a7fafaaeb25 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:11.721292 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c2e32-65fb-4da1-96b6-5a7fafaaeb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:11.724195 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c2e32-65fb-4da1-96b6-5a7fafaaeb25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:11.736288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c2e32-65fb-4da1-96b6-5a7fafaaeb25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:11.738123 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c2e32-65fb-4da1-96b6-5a7fafaaeb25 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:11.754557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-e50c2e32-65fb-4da1-96b6-5a7fafaaeb25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:11.754557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:11.754557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:11.754557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:11.754557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:11.754557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:11.754557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:11.754557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:11.754557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:11.754557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:11.754557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:11.754557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:11.754557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:11.754557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:11.754557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:11.754557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:11.755616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:11.755616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:11.755616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:11.755616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:11.755616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:11.755616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:11.755616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:11.755616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:11.755616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:11.755616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:11.755616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:11.755616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:11.755616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:11.755616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:11.755616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:11.755616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:11.755616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:11.756517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:11.756517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:11.756517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:11.763303 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 403/808] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:11.765331 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c2e32-65fb-4da1-96b6-5a7fafaaeb25 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:11.782686 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:11.886493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f319b8-3e7d-4591-90e0-17a04edb4cda None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:11.904538 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c2e32-65fb-4da1-96b6-5a7fafaaeb25 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:11.921409 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530f1fd-e4ff-41b4-bb2f-e0e651f55aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:11.922052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530f1fd-e4ff-41b4-bb2f-e0e651f55aca None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:11.922333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530f1fd-e4ff-41b4-bb2f-e0e651f55aca None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:11.922608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530f1fd-e4ff-41b4-bb2f-e0e651f55aca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:11.933771 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530f1fd-e4ff-41b4-bb2f-e0e651f55aca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:11.935318 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530f1fd-e4ff-41b4-bb2f-e0e651f55aca None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:11.942256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:11.949277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2cef3-ba9e-4c87-aaf5-d6ae95ff9e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:11.950183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2cef3-ba9e-4c87-aaf5-d6ae95ff9e76 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:11.950664 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2cef3-ba9e-4c87-aaf5-d6ae95ff9e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:11.951160 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2cef3-ba9e-4c87-aaf5-d6ae95ff9e76 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:11.952032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d2cef3-ba9e-4c87-aaf5-d6ae95ff9e76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:11.952385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-4530f1fd-e4ff-41b4-bb2f-e0e651f55aca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:11.952385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:11.952385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:11.952385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:11.952385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:11.952385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:11.952385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:11.952385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:11.952385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:11.952385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:11.952385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:11.952385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:11.952385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:11.952385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:11.952385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:11.952385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:11.953292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:11.953292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:11.953292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:11.953292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:11.953292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:11.953292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:11.953292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:11.953292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:11.953292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:11.953292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:11.953292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:11.953292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:11.953292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:11.953292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:11.953292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:11.953292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:11.953292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:11.954175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:11.954175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:11.954175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:11.955023 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d2cef3-ba9e-4c87-aaf5-d6ae95ff9e76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:11.956646 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d2cef3-ba9e-4c87-aaf5-d6ae95ff9e76 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:11.958085 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 406/809] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:11.959426 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d7c5f-d782-4a1f-a525-6785fc613b55 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:11.959716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d7c5f-d782-4a1f-a525-6785fc613b55 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:11.960445 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d7c5f-d782-4a1f-a525-6785fc613b55 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:11.961877 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 407/810] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:11.962589 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d7c5f-d782-4a1f-a525-6785fc613b55 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:11.973051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 404/811] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:11.979789 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58e681-2dc8-4012-a00e-13adfb9e0a01 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:11.980411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58e681-2dc8-4012-a00e-13adfb9e0a01 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:11.980957 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58e681-2dc8-4012-a00e-13adfb9e0a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:12.006736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-8c58e681-2dc8-4012-a00e-13adfb9e0a01 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:12.008880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-8c58e681-2dc8-4012-a00e-13adfb9e0a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:12.078330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d7c5f-d782-4a1f-a525-6785fc613b55 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:12.097858 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:12.105833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 405/812] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:12.107600 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58e681-2dc8-4012-a00e-13adfb9e0a01 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:12.118652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52c014-cb9c-4784-9211-e90a127efa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:12.119388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52c014-cb9c-4784-9211-e90a127efa7d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:12.119723 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52c014-cb9c-4784-9211-e90a127efa7d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:12.119996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52c014-cb9c-4784-9211-e90a127efa7d None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:12.120864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52c014-cb9c-4784-9211-e90a127efa7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:12.121085 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52c014-cb9c-4784-9211-e90a127efa7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:12.122510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52c014-cb9c-4784-9211-e90a127efa7d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:12.141420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 408/813] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:12.142950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52c014-cb9c-4784-9211-e90a127efa7d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:12.227726 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58e681-2dc8-4012-a00e-13adfb9e0a01 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:12.237968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:12.270401 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbfdb3-cd94-4ae8-b3ec-fb242c5d9bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:12.274025 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbfdb3-cd94-4ae8-b3ec-fb242c5d9bcf None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:12.274521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbfdb3-cd94-4ae8-b3ec-fb242c5d9bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:12.274968 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbfdb3-cd94-4ae8-b3ec-fb242c5d9bcf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:12.284670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fbfdb3-cd94-4ae8-b3ec-fb242c5d9bcf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:12.286466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fbfdb3-cd94-4ae8-b3ec-fb242c5d9bcf None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:12.300863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-78fbfdb3-cd94-4ae8-b3ec-fb242c5d9bcf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:06:12.300863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:12.300863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:12.300863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:12.300863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:12.300863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:12.300863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:12.300863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:12.300863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:12.300863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:12.300863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:12.300863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:12.300863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:06:12.300863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:06:12.300863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:06:12.300863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:12.301848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:12.302813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:12.302813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:12.302813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:06:12.302813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:06:12.302813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:06:12.302813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:06:12.302813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:06:12.302813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:06:12.302813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:06:12.302813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:12.305945 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 406/814] 10.222.0.78 () {56 vars in 1093 bytes} [Mon Sep 19 11:06:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:12.311905 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbfdb3-cd94-4ae8-b3ec-fb242c5d9bcf None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:12.349836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52c014-cb9c-4784-9211-e90a127efa7d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:12.403613 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:12.416615 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b24bf-acd5-4d22-926f-a154d8a09464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:12.417273 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b24bf-acd5-4d22-926f-a154d8a09464 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:12.417556 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b24bf-acd5-4d22-926f-a154d8a09464 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:12.417830 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b24bf-acd5-4d22-926f-a154d8a09464 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:12.435705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b24bf-acd5-4d22-926f-a154d8a09464 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:12.437218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b24bf-acd5-4d22-926f-a154d8a09464 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:12.453342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-e99b24bf-acd5-4d22-926f-a154d8a09464 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:06:12.453342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:12.453342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:12.453342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:12.453342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:12.453342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:12.453342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:12.453342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:12.453342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:12.453342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:12.453342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:12.453342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:12.453342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:06:12.453342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:06:12.453342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:06:12.453342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:12.454493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:12.455494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:12.455494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:12.455494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:06:12.455494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:06:12.455494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:06:12.455494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:06:12.455494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:06:12.455494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:06:12.455494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:06:12.455494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:12.455494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 409/815] 10.222.0.78 () {56 vars in 1117 bytes} [Mon Sep 19 11:06:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:12.465775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b24bf-acd5-4d22-926f-a154d8a09464 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:12.485002 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbfdb3-cd94-4ae8-b3ec-fb242c5d9bcf None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:12.537564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97d3b6-526d-421c-849f-7ab010c131e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:12.539463 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97d3b6-526d-421c-849f-7ab010c131e7 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:12.540009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97d3b6-526d-421c-849f-7ab010c131e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:12.540428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97d3b6-526d-421c-849f-7ab010c131e7 None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:12.541731 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97d3b6-526d-421c-849f-7ab010c131e7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:12.542510 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97d3b6-526d-421c-849f-7ab010c131e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:12.544409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97d3b6-526d-421c-849f-7ab010c131e7 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:12.571244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 407/816] 10.222.0.78 () {56 vars in 1133 bytes} [Mon Sep 19 11:06:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:12.572017 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97d3b6-526d-421c-849f-7ab010c131e7 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:12.610433 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b24bf-acd5-4d22-926f-a154d8a09464 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:12.613724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:12.649334 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5ea9a-09e9-4e44-bf8b-74bb1fd2c975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:12.650105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5ea9a-09e9-4e44-bf8b-74bb1fd2c975 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:12.650348 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5ea9a-09e9-4e44-bf8b-74bb1fd2c975 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:12.650624 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5ea9a-09e9-4e44-bf8b-74bb1fd2c975 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:12.651449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5ea9a-09e9-4e44-bf8b-74bb1fd2c975 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:12.651693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5ea9a-09e9-4e44-bf8b-74bb1fd2c975 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:12.653230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5ea9a-09e9-4e44-bf8b-74bb1fd2c975 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:12.675238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 410/817] 10.222.0.78 () {56 vars in 1149 bytes} [Mon Sep 19 11:06:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:12.676515 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5ea9a-09e9-4e44-bf8b-74bb1fd2c975 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:12.708093 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97d3b6-526d-421c-849f-7ab010c131e7 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:12.732505 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:12.756337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3fb83-f9c2-4dbe-a222-74c7843ccfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:12.757957 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3fb83-f9c2-4dbe-a222-74c7843ccfdd None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:12.758462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3fb83-f9c2-4dbe-a222-74c7843ccfdd None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:12.759038 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3fb83-f9c2-4dbe-a222-74c7843ccfdd None admin] PATH_INFO: `/v3/services/network` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:12.760023 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3fb83-f9c2-4dbe-a222-74c7843ccfdd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:12.762450 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3fb83-f9c2-4dbe-a222-74c7843ccfdd None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:12.774881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-93b3fb83-f9c2-4dbe-a222-74c7843ccfdd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:06:12.774881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:12.774881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:12.774881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:12.774881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:12.774881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:12.774881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:12.774881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:12.774881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:06:12.774881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:12.774881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:12.774881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:12.774881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:12.774881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:12.774881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:12.774881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:12.774881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:12.775999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:12.775999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:12.775999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:12.775999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:12.775999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:12.775999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:12.775999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:12.775999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:12.775999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:12.775999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:12.775999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:12.775999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:12.775999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:12.775999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:12.775999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:12.775999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:12.776833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:12.776833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:12.776833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:12.776833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:12.776833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:12.776833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:12.776833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:12.776833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:12.776833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:12.776833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:12.776833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:12.776833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:12.776833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:06:12.776833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:12.784981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 408/818] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:12 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:12.784981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3fb83-f9c2-4dbe-a222-74c7843ccfdd None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:12.798027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5ea9a-09e9-4e44-bf8b-74bb1fd2c975 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:12.839793 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61674b-dd35-4c1c-8a29-3f49decb8ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:12.841099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61674b-dd35-4c1c-8a29-3f49decb8ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:12.841635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61674b-dd35-4c1c-8a29-3f49decb8ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:12.842077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61674b-dd35-4c1c-8a29-3f49decb8ba0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:12.881386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:12.903037 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3fb83-f9c2-4dbe-a222-74c7843ccfdd None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:12.930465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61674b-dd35-4c1c-8a29-3f49decb8ba0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:12.932974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61674b-dd35-4c1c-8a29-3f49decb8ba0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:12.940374 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41588bcb-c121-4e97-ad1a-64b706aca8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:12.941131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41588bcb-c121-4e97-ad1a-64b706aca8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:12.941508 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41588bcb-c121-4e97-ad1a-64b706aca8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:12.941881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41588bcb-c121-4e97-ad1a-64b706aca8e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:12.955110 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41588bcb-c121-4e97-ad1a-64b706aca8e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:12.956809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41588bcb-c121-4e97-ad1a-64b706aca8e1 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:12.973283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-41588bcb-c121-4e97-ad1a-64b706aca8e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:12.973283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:12.973283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:12.973283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:12.973283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:12.973283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:12.973283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:12.973283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:12.973283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:12.973283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:12.973283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:12.973283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:12.973283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:12.973283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:12.973283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:12.973283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:12.974458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:12.974458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:12.974458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:12.974458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:12.974458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:12.974458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:12.974458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:12.974458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:12.974458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:12.974458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:12.974458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:12.974458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:12.974458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:12.974458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:12.974458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:12.974458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:12.974458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:12.975485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:12.975485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:12.975485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:12.976634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 409/819] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:12.983300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41588bcb-c121-4e97-ad1a-64b706aca8e1 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:13.021874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:13.046478 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 411/820] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:06:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:06:13.047762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61674b-dd35-4c1c-8a29-3f49decb8ba0 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:13.112116 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41588bcb-c121-4e97-ad1a-64b706aca8e1 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:13.137381 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:13.160685 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df36f84-b1c7-4b1c-a4b9-e5a6c5b658c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:13.161599 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df36f84-b1c7-4b1c-a4b9-e5a6c5b658c3 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:13.162186 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df36f84-b1c7-4b1c-a4b9-e5a6c5b658c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:13.162623 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df36f84-b1c7-4b1c-a4b9-e5a6c5b658c3 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:13.163914 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df36f84-b1c7-4b1c-a4b9-e5a6c5b658c3 None admin] RBAC: Adding query filter params (name=network) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:13.164532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df36f84-b1c7-4b1c-a4b9-e5a6c5b658c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:13.166199 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df36f84-b1c7-4b1c-a4b9-e5a6c5b658c3 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:13.172992 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 410/821] 10.222.0.78 () {56 vars in 1112 bytes} [Mon Sep 19 11:06:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:13.176244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df36f84-b1c7-4b1c-a4b9-e5a6c5b658c3 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:13.187548 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61674b-dd35-4c1c-8a29-3f49decb8ba0 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:13.229167 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a7e49-8c2c-4656-a493-b92f2061e6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:13.230384 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a7e49-8c2c-4656-a493-b92f2061e6eb None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:13.230871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a7e49-8c2c-4656-a493-b92f2061e6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:13.231386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a7e49-8c2c-4656-a493-b92f2061e6eb None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:13.232350 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a7e49-8c2c-4656-a493-b92f2061e6eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:13.232853 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a7e49-8c2c-4656-a493-b92f2061e6eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:13.234538 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a7e49-8c2c-4656-a493-b92f2061e6eb None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:13.249757 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 412/822] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:13.251237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a7e49-8c2c-4656-a493-b92f2061e6eb None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:13.287587 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:13.291326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df36f84-b1c7-4b1c-a4b9-e5a6c5b658c3 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:13.325803 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa7d61-dc0e-42f9-90da-62ae9dbda381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:13.326432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa7d61-dc0e-42f9-90da-62ae9dbda381 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:13.326712 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa7d61-dc0e-42f9-90da-62ae9dbda381 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:13.326984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa7d61-dc0e-42f9-90da-62ae9dbda381 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:13.338172 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa7d61-dc0e-42f9-90da-62ae9dbda381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:13.339672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa7d61-dc0e-42f9-90da-62ae9dbda381 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:13.354471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-a6fa7d61-dc0e-42f9-90da-62ae9dbda381 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:13.354471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:13.354471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:13.354471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:13.354471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:13.354471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:13.354471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:13.354471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:13.354471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:13.354471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:13.354471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:13.354471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:13.354471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:13.354471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:13.354471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:13.354471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:13.355676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:13.355676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:13.355676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:13.355676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:13.355676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:13.355676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:13.355676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:13.355676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:13.355676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:13.355676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:13.355676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:13.355676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:13.355676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:13.355676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:13.355676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:13.355676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:13.355676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:13.356576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:13.356576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:13.356576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:13.356967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a7e49-8c2c-4656-a493-b92f2061e6eb None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:13.357094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 411/823] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:13.357094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f184e-961f-4e88-81cc-af302a747ad0 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:13.357094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f184e-961f-4e88-81cc-af302a747ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:13.357294 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f184e-961f-4e88-81cc-af302a747ad0 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:13.358683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 412/824] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:13.359614 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679535d5-ea70-4fd2-a045-c2db3a8e1ed1 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:13.359929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679535d5-ea70-4fd2-a045-c2db3a8e1ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:13.360274 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679535d5-ea70-4fd2-a045-c2db3a8e1ed1 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:13.361473 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 413/825] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:13.362177 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679535d5-ea70-4fd2-a045-c2db3a8e1ed1 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:13.384168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:13.399715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf607e1-eeb4-4c6e-bcba-d29c890bb8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:13.399715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf607e1-eeb4-4c6e-bcba-d29c890bb8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:13.399715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf607e1-eeb4-4c6e-bcba-d29c890bb8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:13.399715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf607e1-eeb4-4c6e-bcba-d29c890bb8e2 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:13.400304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf607e1-eeb4-4c6e-bcba-d29c890bb8e2 None admin] RBAC: Adding query filter params (type=network) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:13.400304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf607e1-eeb4-4c6e-bcba-d29c890bb8e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:13.400304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf607e1-eeb4-4c6e-bcba-d29c890bb8e2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:13.404940 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 413/826] 10.222.0.78 () {56 vars in 1112 bytes} [Mon Sep 19 11:06:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:13.405742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf607e1-eeb4-4c6e-bcba-d29c890bb8e2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:13.473299 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679535d5-ea70-4fd2-a045-c2db3a8e1ed1 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:13.506982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f636819-0c36-46b2-8ee9-1506ccdfd86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:13.508271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f636819-0c36-46b2-8ee9-1506ccdfd86f None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:13.508547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f636819-0c36-46b2-8ee9-1506ccdfd86f None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:13.508819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f636819-0c36-46b2-8ee9-1506ccdfd86f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:13.510993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf607e1-eeb4-4c6e-bcba-d29c890bb8e2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:13.515589 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f636819-0c36-46b2-8ee9-1506ccdfd86f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:13.517028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f636819-0c36-46b2-8ee9-1506ccdfd86f None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:13.528844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-9f636819-0c36-46b2-8ee9-1506ccdfd86f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:13.528844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:13.528844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:13.528844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:13.528844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:13.528844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:13.528844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:13.528844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:13.528844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:13.528844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:13.528844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:13.528844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:13.528844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:06:13.528844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:06:13.528844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:06:13.529784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:06:13.529784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:13.529784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:13.529784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:13.529784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:13.529784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:13.529784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:13.529784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:13.529784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:13.529784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:13.529784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:13.529784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:13.529784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:13.529784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:13.529784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:13.529784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:13.529784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:13.530678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:06:13.530678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:06:13.530678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:06:13.530678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:06:13.530678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:06:13.530678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:06:13.530678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:13.530678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:13.530678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 414/827] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:13.531414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9230e-b84d-4c1a-aaa8-e37c0402d355 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:13.531687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9230e-b84d-4c1a-aaa8-e37c0402d355 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:13.531966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9230e-b84d-4c1a-aaa8-e37c0402d355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:13.545537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01d727-785b-4996-931f-4bce61696f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:13.546371 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01d727-785b-4996-931f-4bce61696f9b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:13.546804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01d727-785b-4996-931f-4bce61696f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:13.547078 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01d727-785b-4996-931f-4bce61696f9b None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:13.547745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01d727-785b-4996-931f-4bce61696f9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:13.548048 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01d727-785b-4996-931f-4bce61696f9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:13.549397 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01d727-785b-4996-931f-4bce61696f9b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:13.549496 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-5fc9230e-b84d-4c1a-aaa8-e37c0402d355 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:13.550657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5fc9230e-b84d-4c1a-aaa8-e37c0402d355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:13.554457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:13.561598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 414/828] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:13.562958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9649f73-805b-4b3d-b3d9-4bc86df1c0a3 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:13.563632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9649f73-805b-4b3d-b3d9-4bc86df1c0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:13.563632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9649f73-805b-4b3d-b3d9-4bc86df1c0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:13.581478 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c9649f73-805b-4b3d-b3d9-4bc86df1c0a3 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:13.582680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c9649f73-805b-4b3d-b3d9-4bc86df1c0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:13.620447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 415/829] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:13.621295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9230e-b84d-4c1a-aaa8-e37c0402d355 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:13.639808 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:13.679655 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 415/830] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:13.679655 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9649f73-805b-4b3d-b3d9-4bc86df1c0a3 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:13.750532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9230e-b84d-4c1a-aaa8-e37c0402d355 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:13.791850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9649f73-805b-4b3d-b3d9-4bc86df1c0a3 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:13.795772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6d1b6-8e99-495d-a882-19c7cbd3b04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:13.796469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6d1b6-8e99-495d-a882-19c7cbd3b04c None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:13.796821 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6d1b6-8e99-495d-a882-19c7cbd3b04c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:13.797165 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6d1b6-8e99-495d-a882-19c7cbd3b04c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:13.797902 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6d1b6-8e99-495d-a882-19c7cbd3b04c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:13.799479 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6d1b6-8e99-495d-a882-19c7cbd3b04c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:13.814353 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:13.828353 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076e80d-2358-453a-84ae-d684f414ee7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:13.829088 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076e80d-2358-453a-84ae-d684f414ee7a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:13.829434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076e80d-2358-453a-84ae-d684f414ee7a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:13.829769 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076e80d-2358-453a-84ae-d684f414ee7a None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:13.830542 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076e80d-2358-453a-84ae-d684f414ee7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:13.830931 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076e80d-2358-453a-84ae-d684f414ee7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:13.832447 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076e80d-2358-453a-84ae-d684f414ee7a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:13.844354 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 416/831] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:06:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:13.845218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076e80d-2358-453a-84ae-d684f414ee7a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:13.884838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:13.887143 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 416/832] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:06:13 2022] POST /identity/v3/endpoints => generated 337 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:06:13.888508 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6d1b6-8e99-495d-a882-19c7cbd3b04c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:13.975249 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076e80d-2358-453a-84ae-d684f414ee7a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:14.027355 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fabac7-bccb-40d6-8a48-5b8269be006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:14.028014 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fabac7-bccb-40d6-8a48-5b8269be006b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:14.028298 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fabac7-bccb-40d6-8a48-5b8269be006b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:14.028568 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fabac7-bccb-40d6-8a48-5b8269be006b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:14.031376 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6d1b6-8e99-495d-a882-19c7cbd3b04c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:14.036312 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fabac7-bccb-40d6-8a48-5b8269be006b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:14.037778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fabac7-bccb-40d6-8a48-5b8269be006b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:14.050703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-a4fabac7-bccb-40d6-8a48-5b8269be006b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:14.050703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:14.050703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:14.050703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:14.050703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:14.050703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:14.050703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:14.050703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:14.050703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:14.050703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:14.050703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:14.050703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:14.050703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:06:14.050703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:06:14.050703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:06:14.051982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:06:14.051982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:14.051982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:14.051982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:14.051982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:14.051982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:14.051982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:14.051982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:14.051982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:14.051982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:14.051982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:14.051982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:14.051982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:14.051982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:14.051982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:14.051982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:14.051982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:14.052872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:06:14.052872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:06:14.052872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:06:14.052872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:06:14.052872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:06:14.052872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:06:14.052872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:14.052872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:14.052872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 417/833] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:14.052872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fabac7-bccb-40d6-8a48-5b8269be006b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:14.075445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cafff-0e1c-41d4-9573-4d154285413b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:14.076425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cafff-0e1c-41d4-9573-4d154285413b None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:14.076775 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cafff-0e1c-41d4-9573-4d154285413b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:14.077117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cafff-0e1c-41d4-9573-4d154285413b None admin] PATH_INFO: `/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/5a667fa07b1148998959835d9995ad60/roles/28365abffd3d4ff6874bce2e0c561f1d` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:14.095964 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cafff-0e1c-41d4-9573-4d154285413b None admin] RBAC: Authorizing `identity:create_grant(project_id=a8563e86b7c94ff29e39fb20a7969551, user_id=5a667fa07b1148998959835d9995ad60, role_id=28365abffd3d4ff6874bce2e0c561f1d)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:14.097559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cafff-0e1c-41d4-9573-4d154285413b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:14.157391 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:14.165260 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 417/834] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:06:13 2022] PUT /identity/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/5a667fa07b1148998959835d9995ad60/roles/28365abffd3d4ff6874bce2e0c561f1d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:06:14.166093 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cafff-0e1c-41d4-9573-4d154285413b None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:14.166265 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fabac7-bccb-40d6-8a48-5b8269be006b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:14.204985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f97b5-f069-4f81-8683-3c3f84e3983a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:14.205659 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f97b5-f069-4f81-8683-3c3f84e3983a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:14.205917 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f97b5-f069-4f81-8683-3c3f84e3983a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:14.206189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f97b5-f069-4f81-8683-3c3f84e3983a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:14.206785 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f97b5-f069-4f81-8683-3c3f84e3983a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:14.208405 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f97b5-f069-4f81-8683-3c3f84e3983a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:14.218863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-804f97b5-f069-4f81-8683-3c3f84e3983a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:06:14.218863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:14.218863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:14.218863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:14.218863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:14.218863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:14.218863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:14.218863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:14.218863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:06:14.218863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:14.218863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:14.218863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:14.218863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:14.218863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:14.218863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:14.218863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:14.218863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:14.221738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:14.221738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:14.221738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:14.221738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:14.221738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:14.221738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:14.221738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:14.221738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:14.221738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:14.221738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:14.221738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:14.221738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:14.221738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:14.221738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:14.221738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:14.221738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:14.222616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:14.222616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:14.222616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:14.222616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:14.222616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:14.222616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:14.222616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:14.222616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:14.222616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:14.222616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:14.222616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:14.222616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:14.222616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:06:14.222616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:14.222616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 418/835] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:06:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:14.223455 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f97b5-f069-4f81-8683-3c3f84e3983a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:14.232173 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:14.301648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cafff-0e1c-41d4-9573-4d154285413b None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:14.347064 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08746f-18df-40c0-9b67-be5512bab60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:14.348496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f97b5-f069-4f81-8683-3c3f84e3983a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:14.348623 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08746f-18df-40c0-9b67-be5512bab60e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:14.348791 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08746f-18df-40c0-9b67-be5512bab60e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:14.349055 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08746f-18df-40c0-9b67-be5512bab60e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:14.361496 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08746f-18df-40c0-9b67-be5512bab60e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:06:14.366461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-ba08746f-18df-40c0-9b67-be5512bab60e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:06:14.366461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:14.366461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:14.366461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:14.366461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:14.366461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:14.366461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:14.366461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:14.366461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:14.366461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:14.366461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:14.366461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:14.366461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:06:14.366461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:06:14.366461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:06:14.366461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:06:14.368314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:06:14.368314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:06:14.368314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:06:14.368314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:06:14.368314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:14.368314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:14.368314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:14.368314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:14.368314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:14.368314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:14.368314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:14.368314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:14.368314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:14.368314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:14.368314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:14.368314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:14.369636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:14.369636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:14.369636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:14.369636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:14.369636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:06:14.369636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:06:14.369636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:06:14.369636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:06:14.369636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:06:14.369636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:06:14.369636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:06:14.369636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:14.369636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 418/836] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:06:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:14.369636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08746f-18df-40c0-9b67-be5512bab60e None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:14.391714 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cf366-de18-4c53-b830-94ffc39f5bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:14.392428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cf366-de18-4c53-b830-94ffc39f5bbd None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:14.392794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cf366-de18-4c53-b830-94ffc39f5bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:14.393141 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cf366-de18-4c53-b830-94ffc39f5bbd None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:14.393846 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cf366-de18-4c53-b830-94ffc39f5bbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:14.394159 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cf366-de18-4c53-b830-94ffc39f5bbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:14.395642 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cf366-de18-4c53-b830-94ffc39f5bbd None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:14.409308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 419/837] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:06:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:14.410161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cf366-de18-4c53-b830-94ffc39f5bbd None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:14.434758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:14.484356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08746f-18df-40c0-9b67-be5512bab60e None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:14.486193 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:14.523139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f1bd4-2ddb-475b-a078-45c11bb4d90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:14.523366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cf366-de18-4c53-b830-94ffc39f5bbd None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:14.523771 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f1bd4-2ddb-475b-a078-45c11bb4d90e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:14.524051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f1bd4-2ddb-475b-a078-45c11bb4d90e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:14.524323 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f1bd4-2ddb-475b-a078-45c11bb4d90e None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:14.525075 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f1bd4-2ddb-475b-a078-45c11bb4d90e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:14.525393 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f1bd4-2ddb-475b-a078-45c11bb4d90e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:14.526920 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f1bd4-2ddb-475b-a078-45c11bb4d90e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:14.536651 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 419/838] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:06:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:14.537656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f1bd4-2ddb-475b-a078-45c11bb4d90e None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:14.559327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a945e6-283e-4882-8d60-0e3794ba1e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:14.559935 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a945e6-283e-4882-8d60-0e3794ba1e1a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:14.560213 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a945e6-283e-4882-8d60-0e3794ba1e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:14.560479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a945e6-283e-4882-8d60-0e3794ba1e1a None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:14.561139 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a945e6-283e-4882-8d60-0e3794ba1e1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:14.561437 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a945e6-283e-4882-8d60-0e3794ba1e1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:14.562809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a945e6-283e-4882-8d60-0e3794ba1e1a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:14.575438 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 420/839] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:06:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:14.575438 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a945e6-283e-4882-8d60-0e3794ba1e1a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:14.650791 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f1bd4-2ddb-475b-a078-45c11bb4d90e None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:14.656694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:14.679934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a945e6-283e-4882-8d60-0e3794ba1e1a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:14.690364 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f536597-256c-4ea6-a794-60277073a145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:14.691086 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f536597-256c-4ea6-a794-60277073a145 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:14.691461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f536597-256c-4ea6-a794-60277073a145 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:14.691834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f536597-256c-4ea6-a794-60277073a145 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:14.692608 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f536597-256c-4ea6-a794-60277073a145 None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=aba1968529574fe5b4c5d114c874f428) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:14.693121 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f536597-256c-4ea6-a794-60277073a145 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:14.694705 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f536597-256c-4ea6-a794-60277073a145 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:14.711262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 420/840] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:06:14 2022] GET /identity/v3/role_assignments?user.id=aba1968529574fe5b4c5d114c874f428&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:14.711262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f536597-256c-4ea6-a794-60277073a145 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:14.727716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd6751-aa91-4578-8281-2b6ca356e2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:14.728462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd6751-aa91-4578-8281-2b6ca356e2fb None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:14.728813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd6751-aa91-4578-8281-2b6ca356e2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:14.729152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd6751-aa91-4578-8281-2b6ca356e2fb None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:14.729976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd6751-aa91-4578-8281-2b6ca356e2fb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:14.730456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd6751-aa91-4578-8281-2b6ca356e2fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:14.731997 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd6751-aa91-4578-8281-2b6ca356e2fb None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:14.747356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 421/841] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:06:14 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:14.771653 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:14.860932 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f536597-256c-4ea6-a794-60277073a145 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:14.901847 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13523bfd-809d-41a0-ac11-6b9fb6ba7727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:14.902510 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13523bfd-809d-41a0-ac11-6b9fb6ba7727 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:14.902792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13523bfd-809d-41a0-ac11-6b9fb6ba7727 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:14.903061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13523bfd-809d-41a0-ac11-6b9fb6ba7727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:14.916288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13523bfd-809d-41a0-ac11-6b9fb6ba7727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:14.917775 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13523bfd-809d-41a0-ac11-6b9fb6ba7727 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:14.934519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-13523bfd-809d-41a0-ac11-6b9fb6ba7727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:14.934519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:14.934519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:14.934519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:14.934519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:14.934519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:14.934519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:14.934519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:14.934519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:14.934519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:14.934519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:14.934519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:14.934519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:14.934519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:14.934519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:14.934519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:14.935778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:14.935778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:14.935778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:14.935778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:14.935778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:14.935778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:14.935778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:14.935778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:14.935778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:14.935778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:14.935778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:14.935778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:14.935778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:14.935778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:14.935778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:14.935778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:14.935778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:14.936884 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:14.936884 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:14.936884 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:14.936884 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 421/842] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:14.941666 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd6751-aa91-4578-8281-2b6ca356e2fb None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:15.057329 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd6751-aa91-4578-8281-2b6ca356e2fb None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:15.095088 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b6d1d-fc16-4454-8f6d-b117b10a4d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:15.095746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b6d1d-fc16-4454-8f6d-b117b10a4d43 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:15.096031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b6d1d-fc16-4454-8f6d-b117b10a4d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:15.096309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b6d1d-fc16-4454-8f6d-b117b10a4d43 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:15.096960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b6d1d-fc16-4454-8f6d-b117b10a4d43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:15.097264 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b6d1d-fc16-4454-8f6d-b117b10a4d43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:15.098640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b6d1d-fc16-4454-8f6d-b117b10a4d43 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:15.120039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 422/843] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:15.125658 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13523bfd-809d-41a0-ac11-6b9fb6ba7727 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:15.211862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:15.231206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13523bfd-809d-41a0-ac11-6b9fb6ba7727 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:15.268135 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e413a47-9ac0-48b4-8cf5-0e96f5b55dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:15.268768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e413a47-9ac0-48b4-8cf5-0e96f5b55dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:15.269050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e413a47-9ac0-48b4-8cf5-0e96f5b55dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:15.269324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e413a47-9ac0-48b4-8cf5-0e96f5b55dc5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:15.277131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e413a47-9ac0-48b4-8cf5-0e96f5b55dc5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:15.278688 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e413a47-9ac0-48b4-8cf5-0e96f5b55dc5 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:15.291980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-4e413a47-9ac0-48b4-8cf5-0e96f5b55dc5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:06:15.291980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:15.291980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:15.291980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:15.291980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:15.291980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:15.291980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:15.291980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:15.291980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:15.291980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:15.291980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:15.291980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:15.291980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:06:15.291980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:06:15.291980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:06:15.291980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:15.293043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:15.293983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:15.293983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:15.293983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:06:15.293983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:06:15.293983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:06:15.293983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:06:15.293983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:06:15.293983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:06:15.293983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:06:15.293983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:15.293983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 422/844] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:06:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:15.299203 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b6d1d-fc16-4454-8f6d-b117b10a4d43 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:15.302218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:15.405912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b6d1d-fc16-4454-8f6d-b117b10a4d43 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:15.442965 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25e320-9de9-43e9-b2d5-43a1f11bf87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:15.443618 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25e320-9de9-43e9-b2d5-43a1f11bf87c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:15.443901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25e320-9de9-43e9-b2d5-43a1f11bf87c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:15.444178 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25e320-9de9-43e9-b2d5-43a1f11bf87c None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:15.444961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc25e320-9de9-43e9-b2d5-43a1f11bf87c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:15.445266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc25e320-9de9-43e9-b2d5-43a1f11bf87c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:15.446689 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc25e320-9de9-43e9-b2d5-43a1f11bf87c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:15.465932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 423/845] 10.222.0.78 () {56 vars in 1135 bytes} [Mon Sep 19 11:06:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:15.471545 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e413a47-9ac0-48b4-8cf5-0e96f5b55dc5 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:15.581924 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e413a47-9ac0-48b4-8cf5-0e96f5b55dc5 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:15.617227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bb9c5-4929-4dfa-9b58-036f25983111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:15.617837 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bb9c5-4929-4dfa-9b58-036f25983111 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:15.618111 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bb9c5-4929-4dfa-9b58-036f25983111 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:15.618382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bb9c5-4929-4dfa-9b58-036f25983111 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:15.631011 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:15.687943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067bb9c5-4929-4dfa-9b58-036f25983111 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:15.689367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067bb9c5-4929-4dfa-9b58-036f25983111 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:15.793147 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 423/846] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:06:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:06:15.798509 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25e320-9de9-43e9-b2d5-43a1f11bf87c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:15.863344 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:15.907896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25e320-9de9-43e9-b2d5-43a1f11bf87c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:15.953222 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca22912-4052-4b96-85ca-db6832cfa3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:15.953791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca22912-4052-4b96-85ca-db6832cfa3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:15.954081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca22912-4052-4b96-85ca-db6832cfa3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:15.954361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca22912-4052-4b96-85ca-db6832cfa3e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:15.966041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca22912-4052-4b96-85ca-db6832cfa3e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:15.967520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca22912-4052-4b96-85ca-db6832cfa3e3 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:15.982225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-3ca22912-4052-4b96-85ca-db6832cfa3e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:15.982225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:15.982225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:15.982225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:15.982225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:15.982225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:15.982225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:15.982225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:15.982225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:15.982225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:15.982225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:15.982225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:15.982225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:15.982225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:15.982225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:15.982225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:15.983255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:15.983255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:15.983255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:15.983255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:15.983255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:15.983255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:15.983255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:15.983255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:15.983255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:15.983255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:15.983255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:15.983255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:15.983255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:15.983255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:15.983255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:15.983255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:15.983255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:15.984159 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:15.984159 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:15.984159 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:15.984159 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 424/847] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:15.992313 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bb9c5-4929-4dfa-9b58-036f25983111 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:16.038390 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:16.099487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bb9c5-4929-4dfa-9b58-036f25983111 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:16.135996 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1388e-d0d9-4eda-b7fb-e565353a43c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:16.136616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1388e-d0d9-4eda-b7fb-e565353a43c9 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:16.137002 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1388e-d0d9-4eda-b7fb-e565353a43c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:16.137175 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1388e-d0d9-4eda-b7fb-e565353a43c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:16.137822 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1388e-d0d9-4eda-b7fb-e565353a43c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:16.138135 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1388e-d0d9-4eda-b7fb-e565353a43c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:16.139681 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1388e-d0d9-4eda-b7fb-e565353a43c9 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:16.152404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 424/848] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:16.157948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca22912-4052-4b96-85ca-db6832cfa3e3 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:16.266237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca22912-4052-4b96-85ca-db6832cfa3e3 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:16.281270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:16.302280 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf550b3-a733-49a6-a400-02704528a9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:16.302905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf550b3-a733-49a6-a400-02704528a9b6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:16.303193 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf550b3-a733-49a6-a400-02704528a9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:16.303466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf550b3-a733-49a6-a400-02704528a9b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:16.320177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf550b3-a733-49a6-a400-02704528a9b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:16.320177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf550b3-a733-49a6-a400-02704528a9b6 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:16.323915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-aaf550b3-a733-49a6-a400-02704528a9b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:16.323915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:16.323915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:16.323915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:16.323915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:16.323915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:16.323915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:16.323915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:16.323915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:16.323915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:16.323915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:16.323915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:16.323915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:06:16.323915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:06:16.323915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:06:16.324801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:06:16.324801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:16.324801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:16.324801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:16.324801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:16.324801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:16.324801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:16.324801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:16.324801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:16.324801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:16.324801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:16.324801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:16.324801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:16.324801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:16.324801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:16.324801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:16.324801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:16.325680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:06:16.325680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:06:16.325680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:06:16.325680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:06:16.325680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:06:16.325680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:06:16.325680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:16.325680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:16.325680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 425/849] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:16.330886 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1388e-d0d9-4eda-b7fb-e565353a43c9 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:16.372587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce833de-00bb-4d09-950b-a95aa9ea7e82 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:16.372922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce833de-00bb-4d09-950b-a95aa9ea7e82 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:16.373132 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce833de-00bb-4d09-950b-a95aa9ea7e82 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:16.374575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 426/850] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:16.380193 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4bdba-1617-4cda-a33f-d828749c02ef None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:16.380475 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4bdba-1617-4cda-a33f-d828749c02ef None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:16.380746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4bdba-1617-4cda-a33f-d828749c02ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:16.401887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-2be4bdba-1617-4cda-a33f-d828749c02ef None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:16.403104 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2be4bdba-1617-4cda-a33f-d828749c02ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:16.447277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1388e-d0d9-4eda-b7fb-e565353a43c9 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:16.451558 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:16.480956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 427/851] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:16.492395 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca879fb0-2a98-4c73-92ba-977f1bf275bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:16.493056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca879fb0-2a98-4c73-92ba-977f1bf275bf None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:16.493337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca879fb0-2a98-4c73-92ba-977f1bf275bf None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:16.493606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca879fb0-2a98-4c73-92ba-977f1bf275bf None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:16.494296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca879fb0-2a98-4c73-92ba-977f1bf275bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:16.494651 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca879fb0-2a98-4c73-92ba-977f1bf275bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:16.496170 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca879fb0-2a98-4c73-92ba-977f1bf275bf None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:16.510087 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 425/852] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:06:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:16.515792 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4bdba-1617-4cda-a33f-d828749c02ef None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:16.623054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4bdba-1617-4cda-a33f-d828749c02ef None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:16.624615 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:16.662866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494cf37-6127-4a81-879a-c2547e99d20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:16.664037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494cf37-6127-4a81-879a-c2547e99d20d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:16.664314 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494cf37-6127-4a81-879a-c2547e99d20d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:16.664585 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494cf37-6127-4a81-879a-c2547e99d20d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:16.665311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494cf37-6127-4a81-879a-c2547e99d20d None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=cf462b2aa3fc4e2ca58851549cf0c644) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:16.665614 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494cf37-6127-4a81-879a-c2547e99d20d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:16.667025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494cf37-6127-4a81-879a-c2547e99d20d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:16.673577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 428/853] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:06:16 2022] GET /identity/v3/role_assignments?user.id=cf462b2aa3fc4e2ca58851549cf0c644&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:16.763105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca879fb0-2a98-4c73-92ba-977f1bf275bf None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:16.878008 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca879fb0-2a98-4c73-92ba-977f1bf275bf None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:16.917196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd025c-950c-4f22-a658-885f9655013d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:16.917836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd025c-950c-4f22-a658-885f9655013d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:16.918111 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd025c-950c-4f22-a658-885f9655013d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:16.918374 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd025c-950c-4f22-a658-885f9655013d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:16.931902 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd025c-950c-4f22-a658-885f9655013d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:06:16.936665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-a9dd025c-950c-4f22-a658-885f9655013d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:06:16.936665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:16.936665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:16.936665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:16.936665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:16.936665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:16.936665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:16.936665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:16.936665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:16.936665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:16.936665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:16.936665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:16.936665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:06:16.936665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:06:16.936665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:06:16.936665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:06:16.937598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:06:16.937598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:06:16.937598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:06:16.937598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:06:16.937598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:16.937598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:16.937598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:16.937598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:16.937598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:16.937598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:16.937598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:16.937598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:16.937598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:16.937598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:16.937598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:16.937598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:16.938446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:16.938446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:16.938446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:16.938446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:16.938446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:06:16.938446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:06:16.938446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:06:16.938446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:06:16.938446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:06:16.938446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:06:16.938446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:06:16.938446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:16.940938 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 426/854] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:06:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:16.944090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494cf37-6127-4a81-879a-c2547e99d20d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:17.061777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494cf37-6127-4a81-879a-c2547e99d20d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:17.097024 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f83a2-61e9-4132-bc5a-9518829dc59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:17.097639 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f83a2-61e9-4132-bc5a-9518829dc59a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:17.097918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f83a2-61e9-4132-bc5a-9518829dc59a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:17.098187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f83a2-61e9-4132-bc5a-9518829dc59a None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:17.098848 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f83a2-61e9-4132-bc5a-9518829dc59a None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:17.099149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f83a2-61e9-4132-bc5a-9518829dc59a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:17.100525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f83a2-61e9-4132-bc5a-9518829dc59a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:17.106798 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2be094-6446-4d20-9e09-5702acaa5b9e None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:17.107091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2be094-6446-4d20-9e09-5702acaa5b9e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:17.107397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2be094-6446-4d20-9e09-5702acaa5b9e None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:17.109774 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 427/856] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:17.109896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 429/856] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:06:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:17.115104 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe788571-b1e4-4a9c-b1e8-37861e4505ae None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:17.115426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe788571-b1e4-4a9c-b1e8-37861e4505ae None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:17.115699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe788571-b1e4-4a9c-b1e8-37861e4505ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:17.119716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f83a2-61e9-4132-bc5a-9518829dc59a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:17.134785 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-fe788571-b1e4-4a9c-b1e8-37861e4505ae None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:17.136056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-fe788571-b1e4-4a9c-b1e8-37861e4505ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:17.152701 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:17.157795 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:17.227260 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 428/857] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:06:17.241821 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f83a2-61e9-4132-bc5a-9518829dc59a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:17.280346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199243cf-e78d-44f5-aa68-b157e6ea0c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:17.281284 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199243cf-e78d-44f5-aa68-b157e6ea0c4b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:17.281810 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fabb73-2baa-44d8-b98e-2684b4e73bce None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:17.281923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199243cf-e78d-44f5-aa68-b157e6ea0c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:17.282188 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fabb73-2baa-44d8-b98e-2684b4e73bce None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:17.282535 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fabb73-2baa-44d8-b98e-2684b4e73bce None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:17.282708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199243cf-e78d-44f5-aa68-b157e6ea0c4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:17.284300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 429/858] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:06:17.289822 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dcbfd-e4bb-439d-830a-1edff95c7011 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:17.290196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dcbfd-e4bb-439d-830a-1edff95c7011 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:17.290548 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dcbfd-e4bb-439d-830a-1edff95c7011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:17.296998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199243cf-e78d-44f5-aa68-b157e6ea0c4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:17.298453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199243cf-e78d-44f5-aa68-b157e6ea0c4b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:17.316471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-474dcbfd-e4bb-439d-830a-1edff95c7011 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:17.317999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-474dcbfd-e4bb-439d-830a-1edff95c7011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:17.319075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-199243cf-e78d-44f5-aa68-b157e6ea0c4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:17.319075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:17.319075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:17.319075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:17.319075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:17.319075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:17.319075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:17.319075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:17.319075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:17.319075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:17.319075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:17.319075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:17.319075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:17.319075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:17.319075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:17.319075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:17.320863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:17.320863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:17.320863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:17.320863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:17.320863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:17.320863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:17.320863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:17.320863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:17.320863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:17.320863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:17.320863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:17.320863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:17.320863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:17.320863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:17.320863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:17.320863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:17.320863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:17.321736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:17.321736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:17.321736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:17.321736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 430/859] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:17.328155 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199243cf-e78d-44f5-aa68-b157e6ea0c4b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:17.414037 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 430/860] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:17.425798 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:17.446208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199243cf-e78d-44f5-aa68-b157e6ea0c4b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:17.481794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e5a9e-72a2-47bc-9601-1abbab45aa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:17.482404 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e5a9e-72a2-47bc-9601-1abbab45aa66 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:17.482682 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e5a9e-72a2-47bc-9601-1abbab45aa66 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:17.482995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e5a9e-72a2-47bc-9601-1abbab45aa66 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:17.483719 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e5a9e-72a2-47bc-9601-1abbab45aa66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:17.484026 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e5a9e-72a2-47bc-9601-1abbab45aa66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:17.485382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e5a9e-72a2-47bc-9601-1abbab45aa66 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:17.500702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 431/861] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:17.502353 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dcbfd-e4bb-439d-830a-1edff95c7011 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:17.503268 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e5a9e-72a2-47bc-9601-1abbab45aa66 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:17.593191 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:17.614344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dcbfd-e4bb-439d-830a-1edff95c7011 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:17.619426 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e5a9e-72a2-47bc-9601-1abbab45aa66 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:17.653829 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a17a21-d4b7-4eba-8440-eab0f648cda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:17.654809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a17a21-d4b7-4eba-8440-eab0f648cda7 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:17.655367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a17a21-d4b7-4eba-8440-eab0f648cda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:17.655855 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a17a21-d4b7-4eba-8440-eab0f648cda7 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:17.656715 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a17a21-d4b7-4eba-8440-eab0f648cda7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:17.657293 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7162ed-c20a-4206-b274-134fb4159794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:17.657926 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7162ed-c20a-4206-b274-134fb4159794 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:17.658201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7162ed-c20a-4206-b274-134fb4159794 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:17.658471 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7162ed-c20a-4206-b274-134fb4159794 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:17.659219 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a17a21-d4b7-4eba-8440-eab0f648cda7 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:17.666450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7162ed-c20a-4206-b274-134fb4159794 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:17.667912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7162ed-c20a-4206-b274-134fb4159794 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:17.670581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:17.680627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-6f7162ed-c20a-4206-b274-134fb4159794 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:06:17.680627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:17.680627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:17.680627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:17.680627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:17.680627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:17.680627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:17.680627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:17.680627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:17.680627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:17.680627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:17.680627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:17.680627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:06:17.680627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:06:17.680627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:06:17.680627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:17.681581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:17.682552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:17.682552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:17.682552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:06:17.682552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:06:17.682552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:06:17.682552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:06:17.682552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:06:17.682552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:06:17.682552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:06:17.682552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:17.686704 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 432/862] 10.222.0.78 () {56 vars in 1117 bytes} [Mon Sep 19 11:06:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:17.688309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7162ed-c20a-4206-b274-134fb4159794 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:17.730867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 431/863] 10.222.0.78 () {60 vars in 1141 bytes} [Mon Sep 19 11:06:17 2022] POST /identity/v3/services => generated 251 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:06:17.732248 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a17a21-d4b7-4eba-8440-eab0f648cda7 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:17.808302 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7162ed-c20a-4206-b274-134fb4159794 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:17.860681 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a17a21-d4b7-4eba-8440-eab0f648cda7 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:17.865739 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e608b-9a71-43cd-ab8f-ad383d622be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:17.866919 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e608b-9a71-43cd-ab8f-ad383d622be9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:17.867232 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e608b-9a71-43cd-ab8f-ad383d622be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:17.867515 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e608b-9a71-43cd-ab8f-ad383d622be9 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:17.868262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e608b-9a71-43cd-ab8f-ad383d622be9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:17.868571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e608b-9a71-43cd-ab8f-ad383d622be9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:17.870708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e608b-9a71-43cd-ab8f-ad383d622be9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:17.897722 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 433/864] 10.222.0.78 () {56 vars in 1149 bytes} [Mon Sep 19 11:06:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:17.899184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e608b-9a71-43cd-ab8f-ad383d622be9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:17.910296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00919452-a85d-46a4-88e2-cb633ab757c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:17.911705 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00919452-a85d-46a4-88e2-cb633ab757c9 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:17.911705 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00919452-a85d-46a4-88e2-cb633ab757c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:17.913415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00919452-a85d-46a4-88e2-cb633ab757c9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:17.919274 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00919452-a85d-46a4-88e2-cb633ab757c9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:17.919274 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00919452-a85d-46a4-88e2-cb633ab757c9 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:17.930027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-00919452-a85d-46a4-88e2-cb633ab757c9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:06:17.930027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:17.930027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:17.930027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:17.930027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:17.930027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:17.930027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:17.930027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:17.930027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:06:17.930027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:17.930027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:17.930027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:17.930027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:17.930027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:17.930027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:17.930027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:17.931591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:17.931591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:17.931591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:17.931591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:17.931591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:17.931591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:17.931591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:17.931591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:17.931591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:17.931591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:17.931591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:17.931591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:17.931591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:17.931591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:17.931591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:17.931591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:17.931591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:17.932696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:17.932696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:17.932696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:17.932696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:17.932696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:17.932696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:17.932696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:17.932696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:17.932696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:17.932696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:17.932696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:17.932696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:17.932696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:06:17.932696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:17.932696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 432/865] 10.222.0.78 () {56 vars in 1132 bytes} [Mon Sep 19 11:06:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:17.939431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00919452-a85d-46a4-88e2-cb633ab757c9 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:17.951522 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:17.986572 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:18.050054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e608b-9a71-43cd-ab8f-ad383d622be9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:18.066084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00919452-a85d-46a4-88e2-cb633ab757c9 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:18.096694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00323320-a4e1-4737-b127-882204acf585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:18.097559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00323320-a4e1-4737-b127-882204acf585 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:18.097842 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00323320-a4e1-4737-b127-882204acf585 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:18.098115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00323320-a4e1-4737-b127-882204acf585 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:18.104465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0229624-6090-4a11-beea-d9818125204c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:18.105391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0229624-6090-4a11-beea-d9818125204c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:18.106434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0229624-6090-4a11-beea-d9818125204c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:18.106930 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0229624-6090-4a11-beea-d9818125204c None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:18.107809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0229624-6090-4a11-beea-d9818125204c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:18.108316 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0229624-6090-4a11-beea-d9818125204c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:18.112550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0229624-6090-4a11-beea-d9818125204c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:18.120877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 433/866] 10.222.0.78 () {56 vars in 1126 bytes} [Mon Sep 19 11:06:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:18.126190 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0229624-6090-4a11-beea-d9818125204c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:18.196430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00323320-a4e1-4737-b127-882204acf585 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:18.198229 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00323320-a4e1-4737-b127-882204acf585 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:18.202180 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:18.249463 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0229624-6090-4a11-beea-d9818125204c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:18.259456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:18.297049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e9e0a-7088-46b8-bfb4-ec5a688d2f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:18.298087 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e9e0a-7088-46b8-bfb4-ec5a688d2f25 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:18.298372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e9e0a-7088-46b8-bfb4-ec5a688d2f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:18.298643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e9e0a-7088-46b8-bfb4-ec5a688d2f25 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:18.299544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e9e0a-7088-46b8-bfb4-ec5a688d2f25 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:18.299862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e9e0a-7088-46b8-bfb4-ec5a688d2f25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:18.301499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e9e0a-7088-46b8-bfb4-ec5a688d2f25 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:18.308784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 434/867] 10.222.0.78 () {56 vars in 1126 bytes} [Mon Sep 19 11:06:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:18.309622 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 434/868] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:06:17 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:06:18.317598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e9e0a-7088-46b8-bfb4-ec5a688d2f25 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:18.436399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e9e0a-7088-46b8-bfb4-ec5a688d2f25 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:18.470614 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:18.478039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280d16c-dffa-4c20-9f37-669e174e9a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:18.478749 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280d16c-dffa-4c20-9f37-669e174e9a3f None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:18.479068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280d16c-dffa-4c20-9f37-669e174e9a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:18.485988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280d16c-dffa-4c20-9f37-669e174e9a3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:18.490631 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280d16c-dffa-4c20-9f37-669e174e9a3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:18.492149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280d16c-dffa-4c20-9f37-669e174e9a3f None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:18.510337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-d280d16c-dffa-4c20-9f37-669e174e9a3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:18.510337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:18.510337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:18.510337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:18.510337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:18.510337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:18.510337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:18.510337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:18.510337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:18.510337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:18.510337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:18.510337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:18.510337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:18.510337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:18.510337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:18.510337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:18.511397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:18.511397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:18.511397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:18.511397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:18.511397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:18.511397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:18.511397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:18.511397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:18.511397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:18.511397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:18.511397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:18.511397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:18.511397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:18.511397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:18.511397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:18.511397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:18.511397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:18.512867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:18.512867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:18.512867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:18.512867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 435/869] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:18.517146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00323320-a4e1-4737-b127-882204acf585 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:18.623855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00323320-a4e1-4737-b127-882204acf585 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:18.665835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36def5-2c5c-47ff-910d-795c0346594b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:18.666449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36def5-2c5c-47ff-910d-795c0346594b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:18.666728 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36def5-2c5c-47ff-910d-795c0346594b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:18.667001 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36def5-2c5c-47ff-910d-795c0346594b None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:18.667694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36def5-2c5c-47ff-910d-795c0346594b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:18.667995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36def5-2c5c-47ff-910d-795c0346594b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:18.669422 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36def5-2c5c-47ff-910d-795c0346594b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:18.681789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 435/870] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:18.687341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280d16c-dffa-4c20-9f37-669e174e9a3f None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:18.802226 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280d16c-dffa-4c20-9f37-669e174e9a3f None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:18.838546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4e506-4778-4e5e-ad7e-90ae9266cb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:18.839210 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4e506-4778-4e5e-ad7e-90ae9266cb0f None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:18.839498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4e506-4778-4e5e-ad7e-90ae9266cb0f None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:18.839770 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4e506-4778-4e5e-ad7e-90ae9266cb0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:18.847379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4e506-4778-4e5e-ad7e-90ae9266cb0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:18.848829 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4e506-4778-4e5e-ad7e-90ae9266cb0f None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:18.860702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-4db4e506-4778-4e5e-ad7e-90ae9266cb0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:18.860702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:18.860702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:18.860702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:18.860702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:18.860702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:18.860702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:18.860702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:18.860702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:18.860702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:18.860702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:18.860702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:18.860702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:06:18.860702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:06:18.860702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:06:18.862648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:06:18.862648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:18.862648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:18.862648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:18.862648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:18.862648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:18.862648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:18.862648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:18.862648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:18.862648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:18.862648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:18.862648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:18.862648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:18.862648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:18.862648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:18.862648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:18.862648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:18.863946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:06:18.863946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:06:18.863946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:06:18.863946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:06:18.863946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:06:18.863946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:06:18.863946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:18.863946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:18.863946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 436/871] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:18.868187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36def5-2c5c-47ff-910d-795c0346594b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:18.886153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:18.984986 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36def5-2c5c-47ff-910d-795c0346594b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:19.020458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b44c90-af93-4e91-99f8-3fa7b03b6c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:19.021070 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b44c90-af93-4e91-99f8-3fa7b03b6c4b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:19.021340 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b44c90-af93-4e91-99f8-3fa7b03b6c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:19.021625 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b44c90-af93-4e91-99f8-3fa7b03b6c4b None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:19.022355 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b44c90-af93-4e91-99f8-3fa7b03b6c4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:19.023326 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b44c90-af93-4e91-99f8-3fa7b03b6c4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:19.024753 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b44c90-af93-4e91-99f8-3fa7b03b6c4b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:19.029138 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2e5fc-1a69-45d7-be9a-7c43383f3c22 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:19.029415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2e5fc-1a69-45d7-be9a-7c43383f3c22 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:19.029691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2e5fc-1a69-45d7-be9a-7c43383f3c22 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:19.031032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 437/872] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:19.036562 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e013b1-35d5-4972-a7ef-61227aa311e7 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:19.036891 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e013b1-35d5-4972-a7ef-61227aa311e7 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:19.037117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e013b1-35d5-4972-a7ef-61227aa311e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:19.037338 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 436/873] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:06:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:19.043219 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b44c90-af93-4e91-99f8-3fa7b03b6c4b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:19.056391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-a6e013b1-35d5-4972-a7ef-61227aa311e7 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:19.057638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a6e013b1-35d5-4972-a7ef-61227aa311e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:19.105652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:19.132516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:19.150069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 438/874] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:19 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:19.163753 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b44c90-af93-4e91-99f8-3fa7b03b6c4b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:19.202886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c819c-b4bf-4408-acad-c28a1661fc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:19.203535 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c819c-b4bf-4408-acad-c28a1661fc02 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:19.203820 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c819c-b4bf-4408-acad-c28a1661fc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:19.204095 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c819c-b4bf-4408-acad-c28a1661fc02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:19.204827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c819c-b4bf-4408-acad-c28a1661fc02 None admin] RBAC: Adding query filter params (role.id=28365abffd3d4ff6874bce2e0c561f1d, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=5a667fa07b1148998959835d9995ad60) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:19.205137 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c819c-b4bf-4408-acad-c28a1661fc02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:19.206568 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c819c-b4bf-4408-acad-c28a1661fc02 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:19.214066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 437/875] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:06:19 2022] GET /identity/v3/role_assignments?user.id=5a667fa07b1148998959835d9995ad60&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=28365abffd3d4ff6874bce2e0c561f1d => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:19.427362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e013b1-35d5-4972-a7ef-61227aa311e7 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:19.535935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e013b1-35d5-4972-a7ef-61227aa311e7 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:19.572932 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c41fe-3d3b-4ef1-b6eb-33daa6e41dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:19.573894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c41fe-3d3b-4ef1-b6eb-33daa6e41dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:19.574358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c41fe-3d3b-4ef1-b6eb-33daa6e41dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:19.574810 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c41fe-3d3b-4ef1-b6eb-33daa6e41dc0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:19.575789 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c41fe-3d3b-4ef1-b6eb-33daa6e41dc0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:19.577340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c41fe-3d3b-4ef1-b6eb-33daa6e41dc0 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:19.587417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-910c41fe-3d3b-4ef1-b6eb-33daa6e41dc0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:06:19.587417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:19.587417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:19.587417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:19.587417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:19.587417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:19.587417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:19.587417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:19.587417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:06:19.587417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:19.587417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:19.587417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:19.587417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:19.587417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:19.587417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:19.587417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:19.587417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:19.588504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:19.588504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:19.588504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:19.588504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:19.588504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:19.588504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:19.588504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:19.588504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:19.588504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:19.588504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:19.588504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:19.588504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:19.588504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:19.588504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:19.588504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:19.588504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:19.589397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:19.589397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:19.589397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:19.589397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:19.589397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:19.589397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:19.589397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:19.589397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:19.589397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:19.589397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:19.589397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:19.589397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:19.589397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:06:19.589397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:19.591802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 439/876] 10.222.0.78 () {56 vars in 1126 bytes} [Mon Sep 19 11:06:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:19.603150 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c819c-b4bf-4408-acad-c28a1661fc02 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:19.706857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c819c-b4bf-4408-acad-c28a1661fc02 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:19.741846 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903e374-ee32-4c33-a254-fc02df742c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:19.742471 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903e374-ee32-4c33-a254-fc02df742c49 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:19.742752 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903e374-ee32-4c33-a254-fc02df742c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:19.743028 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903e374-ee32-4c33-a254-fc02df742c49 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:19.743696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903e374-ee32-4c33-a254-fc02df742c49 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:19.744001 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903e374-ee32-4c33-a254-fc02df742c49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:19.745369 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903e374-ee32-4c33-a254-fc02df742c49 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:19.750712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 438/877] 10.222.0.78 () {56 vars in 1122 bytes} [Mon Sep 19 11:06:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:19.757491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c41fe-3d3b-4ef1-b6eb-33daa6e41dc0 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:19.800202 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:19.868798 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c41fe-3d3b-4ef1-b6eb-33daa6e41dc0 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:19.904491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53aea5-e9a9-4b15-9140-5dcace105363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:19.905103 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53aea5-e9a9-4b15-9140-5dcace105363 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:19.905383 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53aea5-e9a9-4b15-9140-5dcace105363 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:19.905655 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53aea5-e9a9-4b15-9140-5dcace105363 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:19.906279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53aea5-e9a9-4b15-9140-5dcace105363 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:19.906578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53aea5-e9a9-4b15-9140-5dcace105363 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:19.907968 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53aea5-e9a9-4b15-9140-5dcace105363 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:19.912879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 440/878] 10.222.0.78 () {56 vars in 1122 bytes} [Mon Sep 19 11:06:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:20.137837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d0fa7-ca5f-41e0-ab4c-5d0f23765d5c None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:20.138162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d0fa7-ca5f-41e0-ab4c-5d0f23765d5c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:20.138398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d0fa7-ca5f-41e0-ab4c-5d0f23765d5c None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:20.140094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 439/879] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:20.145664 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d013a-b033-4577-a0ed-4342ba04060e None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:20.152540 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d013a-b033-4577-a0ed-4342ba04060e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:20.152540 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d013a-b033-4577-a0ed-4342ba04060e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:20.165638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-2b2d013a-b033-4577-a0ed-4342ba04060e None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:20.166851 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2b2d013a-b033-4577-a0ed-4342ba04060e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:20.243330 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 441/880] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:20.510090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d0fa7-ca5f-41e0-ab4c-5d0f23765d5c None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:20.513304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:20.625955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d0fa7-ca5f-41e0-ab4c-5d0f23765d5c None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:20.660946 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba87d9b-1f73-492c-8b7d-df99f9e659df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:20.661590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba87d9b-1f73-492c-8b7d-df99f9e659df None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:20.661870 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba87d9b-1f73-492c-8b7d-df99f9e659df None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:20.662143 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba87d9b-1f73-492c-8b7d-df99f9e659df None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:20.662740 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba87d9b-1f73-492c-8b7d-df99f9e659df None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:20.664675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba87d9b-1f73-492c-8b7d-df99f9e659df None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:20.673968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-cba87d9b-1f73-492c-8b7d-df99f9e659df None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:06:20.673968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:20.673968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:20.673968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:20.673968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:20.673968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:20.673968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:20.673968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:20.673968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:06:20.673968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:20.673968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:20.673968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:20.673968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:20.673968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:20.673968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:20.673968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:20.673968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:20.675020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:20.675020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:20.675020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:20.675020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:20.675020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:20.675020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:20.675020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:20.675020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:20.675020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:20.675020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:20.675020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:20.675020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:20.675020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:20.675020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:20.675020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:20.675020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:20.675933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:20.675933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:20.675933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:20.675933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:20.675933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:20.675933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:20.675933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:20.675933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:20.675933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:20.675933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:20.675933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:20.675933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:20.675933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:06:20.675933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:20.675933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 440/881] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:06:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:20.684327 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d013a-b033-4577-a0ed-4342ba04060e None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:20.709729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b3520-9f07-4b20-a49f-d33790336235 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:20.710732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b3520-9f07-4b20-a49f-d33790336235 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:20.711009 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b3520-9f07-4b20-a49f-d33790336235 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:20.712398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 441/882] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:06:20.717333 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:20.717859 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb889d-d758-4a30-9579-91d30c19295f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:20.718145 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb889d-d758-4a30-9579-91d30c19295f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:20.718422 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb889d-d758-4a30-9579-91d30c19295f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:20.737411 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-03cb889d-d758-4a30-9579-91d30c19295f None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:20.738624 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-03cb889d-d758-4a30-9579-91d30c19295f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:20.808779 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:20.816282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d013a-b033-4577-a0ed-4342ba04060e None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:20.823547 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 442/883] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:06:20.856994 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f9170-9ab2-4e09-8fd6-1ab9229f9356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:20.857624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f9170-9ab2-4e09-8fd6-1ab9229f9356 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:20.857902 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f9170-9ab2-4e09-8fd6-1ab9229f9356 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:20.858174 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f9170-9ab2-4e09-8fd6-1ab9229f9356 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:20.858818 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f9170-9ab2-4e09-8fd6-1ab9229f9356 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:20.859118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f9170-9ab2-4e09-8fd6-1ab9229f9356 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:20.860616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f9170-9ab2-4e09-8fd6-1ab9229f9356 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:20.865859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 442/884] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:06:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:20.872495 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb889d-d758-4a30-9579-91d30c19295f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:20.977212 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb889d-d758-4a30-9579-91d30c19295f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:21.013302 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20e659-9cf7-4d9b-a162-7bdac9335bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:21.014359 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20e659-9cf7-4d9b-a162-7bdac9335bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:21.014875 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20e659-9cf7-4d9b-a162-7bdac9335bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:21.015386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20e659-9cf7-4d9b-a162-7bdac9335bc4 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:21.016252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20e659-9cf7-4d9b-a162-7bdac9335bc4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:21.016778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20e659-9cf7-4d9b-a162-7bdac9335bc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:21.018347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20e659-9cf7-4d9b-a162-7bdac9335bc4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:21.029133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 443/885] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:06:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:21.032261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f9170-9ab2-4e09-8fd6-1ab9229f9356 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:21.107777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20e659-9cf7-4d9b-a162-7bdac9335bc4 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:21.139262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:21.146258 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f9170-9ab2-4e09-8fd6-1ab9229f9356 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:21.180878 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:21.185086 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b840c-998b-4c91-bff1-630f2c4e6e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:21.185697 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b840c-998b-4c91-bff1-630f2c4e6e7a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:21.186045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b840c-998b-4c91-bff1-630f2c4e6e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:21.186337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b840c-998b-4c91-bff1-630f2c4e6e7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:21.187537 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b840c-998b-4c91-bff1-630f2c4e6e7a None admin] RBAC: Adding query filter params (interface=public, service_id=1e8f95c06edb42e49497867993fb6ec7, region_id=RegionOne) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:21.188211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b840c-998b-4c91-bff1-630f2c4e6e7a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:21.190189 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b840c-998b-4c91-bff1-630f2c4e6e7a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:21.199309 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 443/886] 10.222.0.78 () {56 vars in 1251 bytes} [Mon Sep 19 11:06:21 2022] GET /identity/v3/endpoints?service_id=1e8f95c06edb42e49497867993fb6ec7&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:21.204685 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b840c-998b-4c91-bff1-630f2c4e6e7a None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:21.230918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20e659-9cf7-4d9b-a162-7bdac9335bc4 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:21.276620 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c48fa7-86f6-443e-9bff-932feccead40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:21.278069 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c48fa7-86f6-443e-9bff-932feccead40 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:21.278644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c48fa7-86f6-443e-9bff-932feccead40 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:21.279132 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c48fa7-86f6-443e-9bff-932feccead40 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:21.280636 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c48fa7-86f6-443e-9bff-932feccead40 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:21.282685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c48fa7-86f6-443e-9bff-932feccead40 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:21.329050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b840c-998b-4c91-bff1-630f2c4e6e7a None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:21.364450 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64407c29-7692-4027-a34c-ad67cfeb2c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:21.364979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64407c29-7692-4027-a34c-ad67cfeb2c7d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:21.365263 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64407c29-7692-4027-a34c-ad67cfeb2c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:21.365541 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64407c29-7692-4027-a34c-ad67cfeb2c7d None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:21.366440 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64407c29-7692-4027-a34c-ad67cfeb2c7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:21.367594 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64407c29-7692-4027-a34c-ad67cfeb2c7d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:21.373307 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 444/887] 10.222.0.78 () {56 vars in 1087 bytes} [Mon Sep 19 11:06:21 2022] GET /identity/v3/services => generated 1011 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:06:21.390059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 444/888] 10.222.0.78 () {60 vars in 1141 bytes} [Mon Sep 19 11:06:21 2022] POST /identity/v3/services => generated 269 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:06:21.623785 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dded7ae-31db-4feb-8157-0c2da23dc5a4 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:21.624082 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dded7ae-31db-4feb-8157-0c2da23dc5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:21.624284 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dded7ae-31db-4feb-8157-0c2da23dc5a4 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:21.626122 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 445/889] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:21.631525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342e490-65f9-4c19-91f7-0e5f22588db2 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:21.632552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342e490-65f9-4c19-91f7-0e5f22588db2 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:21.632729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342e490-65f9-4c19-91f7-0e5f22588db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:21.652278 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-7342e490-65f9-4c19-91f7-0e5f22588db2 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:21.653880 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7342e490-65f9-4c19-91f7-0e5f22588db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:21.730231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 445/890] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:22.028319 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dded7ae-31db-4feb-8157-0c2da23dc5a4 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:22.058647 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:22.142538 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dded7ae-31db-4feb-8157-0c2da23dc5a4 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:22.184474 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad7875-227b-4e89-8690-701a5c32240b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:22.184814 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad7875-227b-4e89-8690-701a5c32240b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:22.185043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad7875-227b-4e89-8690-701a5c32240b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:22.186212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f3876-bf66-4b17-8bc8-864374f4b557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:22.186706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 446/891] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:22.186856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f3876-bf66-4b17-8bc8-864374f4b557 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:22.187141 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f3876-bf66-4b17-8bc8-864374f4b557 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:22.187446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f3876-bf66-4b17-8bc8-864374f4b557 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:22.192491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdfd64-9fb8-41eb-b29a-fe4385d7d210 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:22.192782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdfd64-9fb8-41eb-b29a-fe4385d7d210 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:22.193059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdfd64-9fb8-41eb-b29a-fe4385d7d210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:22.201868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f3876-bf66-4b17-8bc8-864374f4b557 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:06:22.208796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-7d7f3876-bf66-4b17-8bc8-864374f4b557 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:06:22.208796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:22.208796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:22.208796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:22.208796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:22.208796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:22.208796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:22.208796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:22.208796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:22.208796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:22.208796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:22.208796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:22.208796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:06:22.208796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:06:22.208796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:06:22.208796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:06:22.210723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:06:22.210723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:06:22.210723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:06:22.210723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:06:22.210723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:22.210723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:22.210723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:22.210723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:22.210723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:22.210723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:22.210723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:22.210723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:22.210723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:22.210723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:22.210723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:22.210723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:22.211674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:22.211674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:22.211674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:22.211674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:22.211674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:06:22.211674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:06:22.211674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:06:22.211674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:06:22.211674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:06:22.211674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:06:22.211674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:06:22.211674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:22.212326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 446/892] 10.222.0.78 () {56 vars in 1120 bytes} [Mon Sep 19 11:06:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:22.212897 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-72bdfd64-9fb8-41eb-b29a-fe4385d7d210 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:22.215444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-72bdfd64-9fb8-41eb-b29a-fe4385d7d210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:22.217981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f3876-bf66-4b17-8bc8-864374f4b557 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:22.261118 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:22.292100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 447/893] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:22.317619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:22.336866 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f3876-bf66-4b17-8bc8-864374f4b557 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:22.375509 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352d3a7-1879-470e-873a-eb29e95699cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:22.376596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352d3a7-1879-470e-873a-eb29e95699cf None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:22.376878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352d3a7-1879-470e-873a-eb29e95699cf None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:22.377151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352d3a7-1879-470e-873a-eb29e95699cf None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:22.377823 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352d3a7-1879-470e-873a-eb29e95699cf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:22.378126 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352d3a7-1879-470e-873a-eb29e95699cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:22.379528 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352d3a7-1879-470e-873a-eb29e95699cf None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:22.388786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 447/894] 10.222.0.78 () {56 vars in 1115 bytes} [Mon Sep 19 11:06:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:22.394674 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdfd64-9fb8-41eb-b29a-fe4385d7d210 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:22.503080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdfd64-9fb8-41eb-b29a-fe4385d7d210 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:22.547078 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d2799-f3f9-4302-9280-b7483f8852e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:22.547682 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d2799-f3f9-4302-9280-b7483f8852e4 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:22.547963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d2799-f3f9-4302-9280-b7483f8852e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:22.548237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d2799-f3f9-4302-9280-b7483f8852e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:22.559942 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d2799-f3f9-4302-9280-b7483f8852e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:22.561382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d2799-f3f9-4302-9280-b7483f8852e4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:22.566118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352d3a7-1879-470e-873a-eb29e95699cf None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:22.579452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-a04d2799-f3f9-4302-9280-b7483f8852e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:22.579452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:22.579452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:22.579452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:22.579452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:22.579452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:22.579452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:22.579452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:22.579452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:22.579452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:22.579452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:22.579452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:22.579452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:22.579452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:22.579452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:22.579452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:22.580491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:22.580491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:22.580491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:22.580491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:22.580491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:22.580491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:22.580491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:22.580491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:22.580491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:22.580491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:22.580491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:22.580491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:22.580491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:22.580491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:22.580491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:22.580491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:22.580491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:22.582114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:22.582114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:22.582114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:22.582114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 448/895] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:22.588464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d2799-f3f9-4302-9280-b7483f8852e4 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:22.623531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:22.684216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352d3a7-1879-470e-873a-eb29e95699cf None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:22.709341 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d2799-f3f9-4302-9280-b7483f8852e4 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:22.729254 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209107a-8b2e-4688-93d8-a8ff9461fcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:22.729874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209107a-8b2e-4688-93d8-a8ff9461fcf0 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:22.730215 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209107a-8b2e-4688-93d8-a8ff9461fcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:22.730487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209107a-8b2e-4688-93d8-a8ff9461fcf0 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:22.731308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0209107a-8b2e-4688-93d8-a8ff9461fcf0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:22.732675 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0209107a-8b2e-4688-93d8-a8ff9461fcf0 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:22.745734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa022e-b425-43fe-b90b-a23a9963d003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:22.751587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa022e-b425-43fe-b90b-a23a9963d003 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:22.751587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa022e-b425-43fe-b90b-a23a9963d003 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:22.751587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa022e-b425-43fe-b90b-a23a9963d003 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:22.751587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa022e-b425-43fe-b90b-a23a9963d003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:22.751587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa022e-b425-43fe-b90b-a23a9963d003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:22.751587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa022e-b425-43fe-b90b-a23a9963d003 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:22.763713 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 449/896] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:22.769136 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa022e-b425-43fe-b90b-a23a9963d003 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:22.818534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 448/897] 10.222.0.78 () {60 vars in 1141 bytes} [Mon Sep 19 11:06:22 2022] POST /identity/v3/services => generated 241 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:06:22.877237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:22.893357 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa022e-b425-43fe-b90b-a23a9963d003 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:22.940189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a7378-0791-4ada-ae59-25490afff92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:22.940849 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a7378-0791-4ada-ae59-25490afff92f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:22.941132 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a7378-0791-4ada-ae59-25490afff92f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:22.941403 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a7378-0791-4ada-ae59-25490afff92f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:22.950221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a7378-0791-4ada-ae59-25490afff92f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:22.951753 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a7378-0791-4ada-ae59-25490afff92f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:22.964952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-678a7378-0791-4ada-ae59-25490afff92f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:06:22.964952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:22.964952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:22.964952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:22.964952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:22.964952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:22.964952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:22.964952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:22.964952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:22.964952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:22.964952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:22.964952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:22.964952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:06:22.964952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:06:22.964952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:06:22.964952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:22.966002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:22.966960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:22.966960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:22.966960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:06:22.966960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:06:22.966960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:06:22.966960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:06:22.966960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:06:22.966960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:06:22.966960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:06:22.966960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:22.966960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 450/898] 10.222.0.78 () {56 vars in 1117 bytes} [Mon Sep 19 11:06:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:22.972716 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209107a-8b2e-4688-93d8-a8ff9461fcf0 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:22.979520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:23.084748 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209107a-8b2e-4688-93d8-a8ff9461fcf0 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:23.125076 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc6229-8506-47cf-9c37-ec6eea500d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:23.125957 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc6229-8506-47cf-9c37-ec6eea500d76 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:23.126237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc6229-8506-47cf-9c37-ec6eea500d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:23.126509 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc6229-8506-47cf-9c37-ec6eea500d76 None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:23.127272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc6229-8506-47cf-9c37-ec6eea500d76 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:23.127578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc6229-8506-47cf-9c37-ec6eea500d76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:23.129048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc6229-8506-47cf-9c37-ec6eea500d76 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:23.148478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 449/899] 10.222.0.78 () {56 vars in 1149 bytes} [Mon Sep 19 11:06:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:23.154156 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a7378-0791-4ada-ae59-25490afff92f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:23.261011 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a7378-0791-4ada-ae59-25490afff92f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:23.295872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb27f3-d5b0-464c-b557-a7d1808cd3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:23.296505 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb27f3-d5b0-464c-b557-a7d1808cd3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:23.296786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb27f3-d5b0-464c-b557-a7d1808cd3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:23.297063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb27f3-d5b0-464c-b557-a7d1808cd3c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:23.320685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:23.370581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb27f3-d5b0-464c-b557-a7d1808cd3c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:23.372045 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb27f3-d5b0-464c-b557-a7d1808cd3c3 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:23.476985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 451/900] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:06:23 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:06:23.482597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc6229-8506-47cf-9c37-ec6eea500d76 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:23.547574 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:23.588570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc6229-8506-47cf-9c37-ec6eea500d76 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:23.634189 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287fe68a-dc99-4144-9965-a7e86cc06bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:23.634766 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fe68a-dc99-4144-9965-a7e86cc06bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:23.635050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fe68a-dc99-4144-9965-a7e86cc06bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:23.635395 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fe68a-dc99-4144-9965-a7e86cc06bb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:23.647995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287fe68a-dc99-4144-9965-a7e86cc06bb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:23.649481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287fe68a-dc99-4144-9965-a7e86cc06bb6 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:23.666382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-287fe68a-dc99-4144-9965-a7e86cc06bb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:23.666382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:23.666382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:23.666382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:23.666382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:23.666382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:23.666382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:23.666382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:23.666382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:23.666382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:23.666382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:23.666382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:23.666382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:23.666382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:23.666382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:23.666382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:23.668970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:23.668970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:23.668970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:23.668970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:23.668970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:23.668970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:23.668970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:23.668970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:23.668970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:23.668970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:23.668970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:23.668970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:23.668970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:23.668970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:23.668970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:23.668970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:23.668970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:23.671151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:23.671151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:23.671151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:23.671151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 450/901] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:23.675195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb27f3-d5b0-464c-b557-a7d1808cd3c3 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:23.738609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69ae63-d51f-4c57-8c2e-2728fdfb39d7 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:23.738932 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69ae63-d51f-4c57-8c2e-2728fdfb39d7 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:23.739150 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69ae63-d51f-4c57-8c2e-2728fdfb39d7 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:23.740530 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 451/902] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:06:23.741767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:23.746973 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f15dbe-1147-4c40-b280-225e29fc0bf3 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:23.747522 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f15dbe-1147-4c40-b280-225e29fc0bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:23.747955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f15dbe-1147-4c40-b280-225e29fc0bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:23.767517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-36f15dbe-1147-4c40-b280-225e29fc0bf3 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:23.768972 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-36f15dbe-1147-4c40-b280-225e29fc0bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:23.780096 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb27f3-d5b0-464c-b557-a7d1808cd3c3 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:23.814977 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1b07b-9984-4b43-a040-59fe6675d219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:23.815649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1b07b-9984-4b43-a040-59fe6675d219 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:23.815932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1b07b-9984-4b43-a040-59fe6675d219 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:23.816206 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1b07b-9984-4b43-a040-59fe6675d219 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:23.816856 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1b07b-9984-4b43-a040-59fe6675d219 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:23.817161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1b07b-9984-4b43-a040-59fe6675d219 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:23.818556 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1b07b-9984-4b43-a040-59fe6675d219 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:23.831555 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 452/903] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:23.832984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3558e3-c8a7-4df0-af97-0da86f963a86 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:23.833274 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3558e3-c8a7-4df0-af97-0da86f963a86 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:23.833545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3558e3-c8a7-4df0-af97-0da86f963a86 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:23.839385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 453/904] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:23.839385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3558e3-c8a7-4df0-af97-0da86f963a86 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:23.849622 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 452/905] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:06:23.851666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68ea07-f3d0-4de8-b0cd-3455fca0fbf8 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:23.851783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68ea07-f3d0-4de8-b0cd-3455fca0fbf8 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:23.852011 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68ea07-f3d0-4de8-b0cd-3455fca0fbf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:23.870315 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-1a68ea07-f3d0-4de8-b0cd-3455fca0fbf8 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:23.871546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1a68ea07-f3d0-4de8-b0cd-3455fca0fbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:23.926546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:23.950601 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 453/906] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:23.952465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3558e3-c8a7-4df0-af97-0da86f963a86 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:23.962726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:23.997313 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86adec-3888-43dd-979f-c98b53e03ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:23.998360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86adec-3888-43dd-979f-c98b53e03ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:23.998906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86adec-3888-43dd-979f-c98b53e03ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:23.999407 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86adec-3888-43dd-979f-c98b53e03ff3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:24.007768 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86adec-3888-43dd-979f-c98b53e03ff3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:24.009542 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86adec-3888-43dd-979f-c98b53e03ff3 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:24.021394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-8e86adec-3888-43dd-979f-c98b53e03ff3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:24.021394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:24.021394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:24.021394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:24.021394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:24.021394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:24.021394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:24.021394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:24.021394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:24.021394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:24.021394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:24.021394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:24.021394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:06:24.021394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:06:24.021394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:06:24.022857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:06:24.022857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:24.022857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:24.022857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:24.022857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:24.022857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:24.022857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:24.022857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:24.022857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:24.022857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:24.022857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:24.022857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:24.022857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:24.022857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:24.022857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:24.022857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:24.022857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:24.023789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:06:24.023789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:06:24.023789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:06:24.023789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:06:24.023789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:06:24.023789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:06:24.023789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:24.023789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:24.026157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 454/907] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:24.031817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68ea07-f3d0-4de8-b0cd-3455fca0fbf8 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:24.142195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86adec-3888-43dd-979f-c98b53e03ff3 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:24.148686 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68ea07-f3d0-4de8-b0cd-3455fca0fbf8 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:24.186977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04233dde-c5af-434a-9b58-b6ad822baf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:24.187636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04233dde-c5af-434a-9b58-b6ad822baf59 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:24.188033 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04233dde-c5af-434a-9b58-b6ad822baf59 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:24.188326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04233dde-c5af-434a-9b58-b6ad822baf59 None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:24.189038 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04233dde-c5af-434a-9b58-b6ad822baf59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:24.189335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04233dde-c5af-434a-9b58-b6ad822baf59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:24.193344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04233dde-c5af-434a-9b58-b6ad822baf59 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:24.203861 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 454/908] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:06:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:24.209526 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04233dde-c5af-434a-9b58-b6ad822baf59 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:24.245555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:24.265084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86adec-3888-43dd-979f-c98b53e03ff3 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:24.302199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d14001-8b05-417e-a3e3-1a7825f50643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:24.303256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d14001-8b05-417e-a3e3-1a7825f50643 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:24.303850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d14001-8b05-417e-a3e3-1a7825f50643 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:24.304365 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d14001-8b05-417e-a3e3-1a7825f50643 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:24.304998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d14001-8b05-417e-a3e3-1a7825f50643 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:24.306371 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d14001-8b05-417e-a3e3-1a7825f50643 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:24.315915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-f4d14001-8b05-417e-a3e3-1a7825f50643 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:06:24.315915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:24.315915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:24.315915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:24.315915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:24.315915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:24.315915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:24.315915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:24.315915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:06:24.315915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:24.315915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:24.315915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:24.315915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:24.315915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:24.315915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:24.315915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:24.315915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:24.316971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:24.316971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:24.316971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:24.316971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:24.316971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:24.316971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:24.316971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:24.316971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:24.316971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:24.316971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:24.316971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:24.316971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:24.316971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:24.316971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:24.316971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:24.316971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:24.320938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:24.320938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:24.320938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:24.320938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:24.320938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:24.320938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:24.320938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:24.320938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:24.320938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:24.320938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:24.320938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:24.320938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:24.320938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:06:24.320938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:24.321679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04233dde-c5af-434a-9b58-b6ad822baf59 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:24.321764 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 455/909] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:06:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:24.321764 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d14001-8b05-417e-a3e3-1a7825f50643 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:24.352225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:24.357308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a21f4a-2145-498b-b8c2-fe64b860c1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:24.358217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a21f4a-2145-498b-b8c2-fe64b860c1bb None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:24.358698 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a21f4a-2145-498b-b8c2-fe64b860c1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:24.359162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a21f4a-2145-498b-b8c2-fe64b860c1bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:24.360137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a21f4a-2145-498b-b8c2-fe64b860c1bb None admin] RBAC: Adding query filter params (role.id=985ecf1b99cb4533a39fad056b155abb, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=5a667fa07b1148998959835d9995ad60) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:24.360448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a21f4a-2145-498b-b8c2-fe64b860c1bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:24.361886 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a21f4a-2145-498b-b8c2-fe64b860c1bb None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:24.367624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 455/910] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:06:24 2022] GET /identity/v3/role_assignments?user.id=5a667fa07b1148998959835d9995ad60&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=985ecf1b99cb4533a39fad056b155abb => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:24.368484 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a21f4a-2145-498b-b8c2-fe64b860c1bb None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:24.465208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d14001-8b05-417e-a3e3-1a7825f50643 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:24.483975 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:24.488040 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a21f4a-2145-498b-b8c2-fe64b860c1bb None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:24.525586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196ef7c-efda-443f-a1fa-5a70c6a05a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:24.526842 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196ef7c-efda-443f-a1fa-5a70c6a05a01 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:24.527326 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196ef7c-efda-443f-a1fa-5a70c6a05a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:24.527851 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196ef7c-efda-443f-a1fa-5a70c6a05a01 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:24.528287 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d29a1-7cdc-4ee9-9c5f-f31592397f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:24.528857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196ef7c-efda-443f-a1fa-5a70c6a05a01 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:24.528976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d29a1-7cdc-4ee9-9c5f-f31592397f68 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:24.529324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d29a1-7cdc-4ee9-9c5f-f31592397f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:24.529676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d29a1-7cdc-4ee9-9c5f-f31592397f68 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:24.530451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d29a1-7cdc-4ee9-9c5f-f31592397f68 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:24.530842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d29a1-7cdc-4ee9-9c5f-f31592397f68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:24.531377 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196ef7c-efda-443f-a1fa-5a70c6a05a01 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:24.532389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d29a1-7cdc-4ee9-9c5f-f31592397f68 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:24.542583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 456/911] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:06:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:24.543726 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d29a1-7cdc-4ee9-9c5f-f31592397f68 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:24.545017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-8196ef7c-efda-443f-a1fa-5a70c6a05a01 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:06:24.545017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:24.545017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:24.545017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:24.545017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:24.545017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:24.545017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:24.545017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:24.545017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:06:24.545017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:24.545017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:24.545017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:24.545017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:24.545017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:24.545017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:24.545017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:24.545928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:24.545928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:24.545928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:24.545928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:24.545928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:24.545928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:24.545928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:24.545928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:24.545928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:24.545928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:24.545928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:24.545928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:24.545928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:24.545928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:24.545928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:24.545928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:24.545928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:24.546992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:24.546992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:24.546992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:24.546992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:24.546992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:24.546992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:24.546992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:24.546992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:24.546992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:24.546992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:24.546992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:24.546992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:24.546992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:06:24.546992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:24.546992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 456/912] 10.222.0.78 () {56 vars in 1132 bytes} [Mon Sep 19 11:06:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:24.552835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196ef7c-efda-443f-a1fa-5a70c6a05a01 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:24.657703 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d29a1-7cdc-4ee9-9c5f-f31592397f68 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:24.667206 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:24.676812 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196ef7c-efda-443f-a1fa-5a70c6a05a01 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:24.694978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033be24-7710-4a04-93d4-0727d2a56f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:24.695611 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033be24-7710-4a04-93d4-0727d2a56f4c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:24.695888 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033be24-7710-4a04-93d4-0727d2a56f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:24.696158 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033be24-7710-4a04-93d4-0727d2a56f4c None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:24.696789 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033be24-7710-4a04-93d4-0727d2a56f4c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:24.697087 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033be24-7710-4a04-93d4-0727d2a56f4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:24.698592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033be24-7710-4a04-93d4-0727d2a56f4c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:24.709043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 457/913] 10.222.0.78 () {56 vars in 1114 bytes} [Mon Sep 19 11:06:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:24.709954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033be24-7710-4a04-93d4-0727d2a56f4c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:24.717922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df1352-cbbf-432a-b4c6-c03d112aa3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:24.718895 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df1352-cbbf-432a-b4c6-c03d112aa3dd None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:24.719429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df1352-cbbf-432a-b4c6-c03d112aa3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:24.719918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df1352-cbbf-432a-b4c6-c03d112aa3dd None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:24.720759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df1352-cbbf-432a-b4c6-c03d112aa3dd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:24.721264 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df1352-cbbf-432a-b4c6-c03d112aa3dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:24.722880 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df1352-cbbf-432a-b4c6-c03d112aa3dd None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:24.728759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 457/914] 10.222.0.78 () {56 vars in 1126 bytes} [Mon Sep 19 11:06:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:24.733861 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df1352-cbbf-432a-b4c6-c03d112aa3dd None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:24.748602 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:24.832007 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033be24-7710-4a04-93d4-0727d2a56f4c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:24.840755 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df1352-cbbf-432a-b4c6-c03d112aa3dd None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:24.870693 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4c9d1-7713-43a4-bcce-6fdeff992073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:24.871761 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4c9d1-7713-43a4-bcce-6fdeff992073 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:24.872323 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4c9d1-7713-43a4-bcce-6fdeff992073 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:24.872998 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4c9d1-7713-43a4-bcce-6fdeff992073 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:24.873904 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4c9d1-7713-43a4-bcce-6fdeff992073 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:24.876039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4c9d1-7713-43a4-bcce-6fdeff992073 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:24.877152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297d314-4b30-40a0-a584-f331c5d114ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:24.877956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297d314-4b30-40a0-a584-f331c5d114ca None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:24.878379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297d314-4b30-40a0-a584-f331c5d114ca None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:24.878791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297d314-4b30-40a0-a584-f331c5d114ca None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:24.879607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297d314-4b30-40a0-a584-f331c5d114ca None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:24.880047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297d314-4b30-40a0-a584-f331c5d114ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:24.881627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297d314-4b30-40a0-a584-f331c5d114ca None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:24.887348 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 458/915] 10.222.0.78 () {56 vars in 1126 bytes} [Mon Sep 19 11:06:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:24.892643 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297d314-4b30-40a0-a584-f331c5d114ca None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:24.916505 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:24.970010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 458/916] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:06:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:06:25.012675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297d314-4b30-40a0-a584-f331c5d114ca None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:25.062004 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7c0f3-1aab-4583-8111-629f858e738f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:25.062729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7c0f3-1aab-4583-8111-629f858e738f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:25.063009 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7c0f3-1aab-4583-8111-629f858e738f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:25.063316 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7c0f3-1aab-4583-8111-629f858e738f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:25.064085 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b7c0f3-1aab-4583-8111-629f858e738f None admin] RBAC: Adding query filter params (interface=public, service_id=59214dbe43f34b9eac3c877b7748372d, region_id=RegionOne) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:25.064450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b7c0f3-1aab-4583-8111-629f858e738f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:25.066126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b7c0f3-1aab-4583-8111-629f858e738f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:25.072667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 459/917] 10.222.0.78 () {56 vars in 1251 bytes} [Mon Sep 19 11:06:24 2022] GET /identity/v3/endpoints?service_id=59214dbe43f34b9eac3c877b7748372d&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:25.078918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4c9d1-7713-43a4-bcce-6fdeff992073 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:25.167238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff889e9-e4b8-447c-a480-d160cbfee2e8 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:25.167529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff889e9-e4b8-447c-a480-d160cbfee2e8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:25.167819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff889e9-e4b8-447c-a480-d160cbfee2e8 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:25.169332 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 460/918] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:06:25.175213 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da52d6-023e-4237-a290-8da1f9c90e7c None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:25.175851 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da52d6-023e-4237-a290-8da1f9c90e7c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:25.176324 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da52d6-023e-4237-a290-8da1f9c90e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:25.196740 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f7da52d6-023e-4237-a290-8da1f9c90e7c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:25.198279 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f7da52d6-023e-4237-a290-8da1f9c90e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:25.198391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:25.220356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4c9d1-7713-43a4-bcce-6fdeff992073 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:25.267017 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967a795-0d0b-4dce-8a57-2278bd8edc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:25.267709 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967a795-0d0b-4dce-8a57-2278bd8edc76 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:25.268009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967a795-0d0b-4dce-8a57-2278bd8edc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:25.268280 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967a795-0d0b-4dce-8a57-2278bd8edc76 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:25.268898 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967a795-0d0b-4dce-8a57-2278bd8edc76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:25.270906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967a795-0d0b-4dce-8a57-2278bd8edc76 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:25.278742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 459/919] 10.222.0.78 () {56 vars in 1087 bytes} [Mon Sep 19 11:06:25 2022] GET /identity/v3/services => generated 1497 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:06:25.291595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 461/920] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:06:25.567695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967a795-0d0b-4dce-8a57-2278bd8edc76 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:25.672459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967a795-0d0b-4dce-8a57-2278bd8edc76 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:25.707079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e771e2-71f3-42e0-8244-56a454f8f816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:25.707754 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e771e2-71f3-42e0-8244-56a454f8f816 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:25.708029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e771e2-71f3-42e0-8244-56a454f8f816 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:25.708294 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e771e2-71f3-42e0-8244-56a454f8f816 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:25.708864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e771e2-71f3-42e0-8244-56a454f8f816 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:25.710818 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e771e2-71f3-42e0-8244-56a454f8f816 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:25.719710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-80e771e2-71f3-42e0-8244-56a454f8f816 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:06:25.719710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:25.719710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:25.719710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:25.719710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:25.719710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:25.719710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:25.719710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:25.719710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:06:25.719710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:25.719710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:25.719710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:25.719710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:25.719710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:25.719710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:25.719710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:25.719710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:25.720730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:25.720730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:25.720730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:25.720730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:25.720730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:25.720730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:25.720730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:25.720730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:25.720730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:25.720730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:25.720730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:25.720730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:25.720730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:25.720730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:25.720730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:25.720730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:25.721573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:25.721573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:25.721573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:25.721573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:25.721573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:25.721573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:25.721573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:25.721573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:25.721573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:25.721573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:25.721573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:25.721573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:25.721573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:06:25.721573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:25.721573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 460/921] 10.222.0.78 () {56 vars in 1126 bytes} [Mon Sep 19 11:06:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:25.726896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da52d6-023e-4237-a290-8da1f9c90e7c None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:25.738606 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:25.834181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da52d6-023e-4237-a290-8da1f9c90e7c None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:25.870200 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae4105-8356-4a52-a442-f8957a2343e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:25.871745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae4105-8356-4a52-a442-f8957a2343e9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:25.872240 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae4105-8356-4a52-a442-f8957a2343e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:25.872719 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae4105-8356-4a52-a442-f8957a2343e9 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:25.873553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae4105-8356-4a52-a442-f8957a2343e9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:25.874073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae4105-8356-4a52-a442-f8957a2343e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:25.875947 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae4105-8356-4a52-a442-f8957a2343e9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:25.887218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 462/922] 10.222.0.78 () {56 vars in 1122 bytes} [Mon Sep 19 11:06:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:25.890214 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e771e2-71f3-42e0-8244-56a454f8f816 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:25.906603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:25.996745 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e771e2-71f3-42e0-8244-56a454f8f816 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:26.036453 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc75cb4-e9dd-4d61-bf59-462b96fc5f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:26.037459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc75cb4-e9dd-4d61-bf59-462b96fc5f04 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:26.038640 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc75cb4-e9dd-4d61-bf59-462b96fc5f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:26.039111 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc75cb4-e9dd-4d61-bf59-462b96fc5f04 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:26.039966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc75cb4-e9dd-4d61-bf59-462b96fc5f04 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:26.040487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc75cb4-e9dd-4d61-bf59-462b96fc5f04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:26.042105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc75cb4-e9dd-4d61-bf59-462b96fc5f04 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:26.052561 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 461/923] 10.222.0.78 () {56 vars in 1122 bytes} [Mon Sep 19 11:06:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:26.055095 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae4105-8356-4a52-a442-f8957a2343e9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:26.142457 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:26.170600 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae4105-8356-4a52-a442-f8957a2343e9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:26.204307 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04f0f1-29ae-4b7c-a8c5-22c1dd161d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:26.205283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04f0f1-29ae-4b7c-a8c5-22c1dd161d8b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:26.205767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04f0f1-29ae-4b7c-a8c5-22c1dd161d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:26.206317 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04f0f1-29ae-4b7c-a8c5-22c1dd161d8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:26.207327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d04f0f1-29ae-4b7c-a8c5-22c1dd161d8b None admin] RBAC: Adding query filter params (interface=public, service_id=435a9b783fa046a49d8f334d745152fb, region_id=RegionOne) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:26.207841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d04f0f1-29ae-4b7c-a8c5-22c1dd161d8b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:26.209414 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d04f0f1-29ae-4b7c-a8c5-22c1dd161d8b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:26.219452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 463/924] 10.222.0.78 () {56 vars in 1251 bytes} [Mon Sep 19 11:06:26 2022] GET /identity/v3/endpoints?service_id=435a9b783fa046a49d8f334d745152fb&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:26.222352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc75cb4-e9dd-4d61-bf59-462b96fc5f04 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:26.306101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:26.332104 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc75cb4-e9dd-4d61-bf59-462b96fc5f04 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:26.368543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f909ba-b6bc-40df-a785-997a277f7347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:26.369519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f909ba-b6bc-40df-a785-997a277f7347 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:26.370020 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f909ba-b6bc-40df-a785-997a277f7347 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:26.370505 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f909ba-b6bc-40df-a785-997a277f7347 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:26.371350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f909ba-b6bc-40df-a785-997a277f7347 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:26.372928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f909ba-b6bc-40df-a785-997a277f7347 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:26.385999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 462/925] 10.222.0.78 () {56 vars in 1087 bytes} [Mon Sep 19 11:06:26 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:06:26.644036 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4c45d-aaf1-4dbd-b00f-dd886682734c None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:26.644227 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4c45d-aaf1-4dbd-b00f-dd886682734c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:26.644503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4c45d-aaf1-4dbd-b00f-dd886682734c None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:26.646056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 464/926] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:26.651983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d20bf-67ed-48a8-877b-8882418429e4 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:26.652268 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d20bf-67ed-48a8-877b-8882418429e4 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:26.652540 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d20bf-67ed-48a8-877b-8882418429e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:26.670982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-ad6d20bf-67ed-48a8-877b-8882418429e4 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:26.672195 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ad6d20bf-67ed-48a8-877b-8882418429e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:26.743126 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 463/927] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:27.021001 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4c45d-aaf1-4dbd-b00f-dd886682734c None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:27.128953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4c45d-aaf1-4dbd-b00f-dd886682734c None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:27.164758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f28b9e-b685-40a1-9aee-9ff9c1cee101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:27.165402 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f28b9e-b685-40a1-9aee-9ff9c1cee101 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:27.165679 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f28b9e-b685-40a1-9aee-9ff9c1cee101 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:27.165944 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f28b9e-b685-40a1-9aee-9ff9c1cee101 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:27.173596 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:27.177392 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f28b9e-b685-40a1-9aee-9ff9c1cee101 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:06:27.182881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-83f28b9e-b685-40a1-9aee-9ff9c1cee101 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:06:27.182881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:27.182881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:27.182881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:27.182881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:27.182881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:27.182881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:27.182881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:27.182881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:27.182881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:27.182881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:27.182881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:27.182881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:06:27.182881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:06:27.182881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:06:27.182881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:06:27.183916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:06:27.183916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:06:27.183916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:06:27.183916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:06:27.183916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:27.183916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:27.183916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:27.183916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:27.183916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:27.183916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:27.183916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:27.183916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:27.183916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:27.183916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:27.183916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:27.183916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:27.185177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:27.185177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:27.185177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:27.185177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:27.185177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:06:27.185177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:06:27.185177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:06:27.185177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:06:27.185177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:06:27.185177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:06:27.185177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:06:27.185177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:27.185177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 465/928] 10.222.0.78 () {56 vars in 1120 bytes} [Mon Sep 19 11:06:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:27.190367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d20bf-67ed-48a8-877b-8882418429e4 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:27.264513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:27.301135 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d20bf-67ed-48a8-877b-8882418429e4 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:27.342744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a40e80-16a9-4a4b-9c72-ee23aee7c90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:27.343413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a40e80-16a9-4a4b-9c72-ee23aee7c90b None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:27.343690 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a40e80-16a9-4a4b-9c72-ee23aee7c90b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:27.343962 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a40e80-16a9-4a4b-9c72-ee23aee7c90b None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:27.344654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a40e80-16a9-4a4b-9c72-ee23aee7c90b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:27.344959 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a40e80-16a9-4a4b-9c72-ee23aee7c90b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:27.346501 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a40e80-16a9-4a4b-9c72-ee23aee7c90b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:27.351734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776535c1-fd81-4e44-bc70-aaee23eb952d None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:27.352723 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776535c1-fd81-4e44-bc70-aaee23eb952d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:27.353128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776535c1-fd81-4e44-bc70-aaee23eb952d None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:27.355024 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 466/929] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:27.358133 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 464/930] 10.222.0.78 () {56 vars in 1115 bytes} [Mon Sep 19 11:06:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:27.364495 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a40e80-16a9-4a4b-9c72-ee23aee7c90b None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:27.364716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a403339-3004-4c6a-881a-a1380e17d690 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:27.364982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a403339-3004-4c6a-881a-a1380e17d690 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:27.365332 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a403339-3004-4c6a-881a-a1380e17d690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:27.389799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-7a403339-3004-4c6a-881a-a1380e17d690 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:27.391805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7a403339-3004-4c6a-881a-a1380e17d690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:27.477664 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 467/931] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:27.494592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a40e80-16a9-4a4b-9c72-ee23aee7c90b None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:27.526232 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:27.534993 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75ae3c-3964-4ec6-8b95-a5567bdac4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:27.538727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75ae3c-3964-4ec6-8b95-a5567bdac4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:27.538727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75ae3c-3964-4ec6-8b95-a5567bdac4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:27.538727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75ae3c-3964-4ec6-8b95-a5567bdac4f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:27.548687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75ae3c-3964-4ec6-8b95-a5567bdac4f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:27.550363 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75ae3c-3964-4ec6-8b95-a5567bdac4f9 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:27.564021 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcb33b-b61d-4d76-853b-8bcd67c4e180 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:27.564215 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcb33b-b61d-4d76-853b-8bcd67c4e180 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:27.564496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcb33b-b61d-4d76-853b-8bcd67c4e180 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:27.566071 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 468/932] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:06:27.567095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-df75ae3c-3964-4ec6-8b95-a5567bdac4f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:27.567095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:27.567095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:27.567095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:27.567095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:27.567095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:27.567095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:27.567095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:27.567095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:27.567095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:27.567095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:27.567095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:27.567095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:27.567095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:27.567095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:27.567095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:27.568236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:27.568236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:27.568236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:27.568236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:27.568236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:27.568236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:27.568236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:27.568236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:27.568236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:27.568236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:27.568236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:27.568236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:27.568236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:27.568236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:27.568236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:27.568236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:27.568236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:27.569929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:27.569929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:27.569929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:27.571481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 465/933] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:27.571638 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1c98d-bdaa-4536-83c0-58adbbbfc6a1 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:27.571833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1c98d-bdaa-4536-83c0-58adbbbfc6a1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:27.572103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1c98d-bdaa-4536-83c0-58adbbbfc6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:27.579216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75ae3c-3964-4ec6-8b95-a5567bdac4f9 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:27.594507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-04d1c98d-bdaa-4536-83c0-58adbbbfc6a1 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:27.595965 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-04d1c98d-bdaa-4536-83c0-58adbbbfc6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:27.686427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 469/934] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:27.702019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75ae3c-3964-4ec6-8b95-a5567bdac4f9 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:27.738867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224beadc-67a7-4a4b-9611-f839a03c478a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:27.739517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224beadc-67a7-4a4b-9611-f839a03c478a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:27.739788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224beadc-67a7-4a4b-9611-f839a03c478a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:27.740055 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224beadc-67a7-4a4b-9611-f839a03c478a None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:27.740698 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224beadc-67a7-4a4b-9611-f839a03c478a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:27.741581 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224beadc-67a7-4a4b-9611-f839a03c478a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:27.742945 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224beadc-67a7-4a4b-9611-f839a03c478a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:27.755788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 466/935] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:27.763094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d1c98d-bdaa-4536-83c0-58adbbbfc6a1 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:27.775990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224beadc-67a7-4a4b-9611-f839a03c478a None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:27.826205 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:27.834871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:27.883721 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d1c98d-bdaa-4536-83c0-58adbbbfc6a1 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:27.899046 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224beadc-67a7-4a4b-9611-f839a03c478a None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:27.927900 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c10fc-a5bb-4317-ae95-24799e9c8e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:27.929128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c10fc-a5bb-4317-ae95-24799e9c8e0c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:27.929409 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c10fc-a5bb-4317-ae95-24799e9c8e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:27.929682 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c10fc-a5bb-4317-ae95-24799e9c8e0c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:27.938823 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c10fc-a5bb-4317-ae95-24799e9c8e0c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:27.940364 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d20f1-82b5-4d70-a7a6-e1cbf8210cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:27.940718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c10fc-a5bb-4317-ae95-24799e9c8e0c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:27.941612 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d20f1-82b5-4d70-a7a6-e1cbf8210cc9 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:27.942100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d20f1-82b5-4d70-a7a6-e1cbf8210cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:27.942567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d20f1-82b5-4d70-a7a6-e1cbf8210cc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:27.943675 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d20f1-82b5-4d70-a7a6-e1cbf8210cc9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:27.946085 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d20f1-82b5-4d70-a7a6-e1cbf8210cc9 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:27.949271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: WARNING keystone.server.flask.common [None req-890d20f1-82b5-4d70-a7a6-e1cbf8210cc9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 19 11:06:27.953959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-935c10fc-a5bb-4317-ae95-24799e9c8e0c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:06:27.953959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:27.953959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:27.953959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:27.953959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:27.953959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:27.953959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:27.953959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:27.953959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:27.953959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:27.953959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:27.953959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:27.953959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:06:27.953959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:06:27.953959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:06:27.953959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:27.954976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:27.955961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:27.955961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:27.955961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:06:27.955961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:06:27.955961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:06:27.955961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:06:27.955961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:06:27.955961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:06:27.955961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:06:27.955961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:27.959373 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 470/936] 10.222.0.78 () {56 vars in 1117 bytes} [Mon Sep 19 11:06:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:27.960443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c10fc-a5bb-4317-ae95-24799e9c8e0c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:27.997912 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.sql.core [None req-890d20f1-82b5-4d70-a7a6-e1cbf8210cc9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 11:06:27.997912 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 19 11:06:27.997912 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [parameters: {'id': 'c0ec220496ce4aa984fc1b78b718ed85', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 19 11:06:27.997912 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81738) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 11:06:28.005727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-890d20f1-82b5-4d70-a7a6-e1cbf8210cc9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c0ec220496ce4aa984fc1b78b718ed85.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c0ec220496ce4aa984fc1b78b718ed85. Sep 19 11:06:28.005727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:28.005727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:06:28.005727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:06:28.005727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:06:28.005727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:06:28.005727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:06:28.005727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:06:28.005727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:06:28.005727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:06:28.005727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:06:28.005727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:06:28.005727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application result.read() Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 19 11:06:28.006605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 19 11:06:28.007598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:06:28.007598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:06:28.007598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 11:06:28.007598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application yield resource Sep 19 11:06:28.007598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:06:28.007598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:06:28.007598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 11:06:28.007598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 11:06:28.007598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 11:06:28.007598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 11:06:28.007598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 11:06:28.007598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise self.value Sep 19 11:06:28.007598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 11:06:28.007598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 11:06:28.007598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 11:06:28.007598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application session.commit() Sep 19 11:06:28.007598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 11:06:28.008477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 11:06:28.008477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 11:06:28.008477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 11:06:28.008477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 11:06:28.008477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self.session.flush() Sep 19 11:06:28.008477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 11:06:28.008477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 11:06:28.008477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 11:06:28.008477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 11:06:28.008477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 11:06:28.008477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application compat.raise_( Sep 19 11:06:28.008477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:06:28.008477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception Sep 19 11:06:28.008477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 11:06:28.008477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 11:06:28.008477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 11:06:28.008477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 11:06:28.009423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 11:06:28.009423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 11:06:28.009423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 11:06:28.009423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 11:06:28.009423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 19 11:06:28.009423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 19 11:06:28.009423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 11:06:28.009423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 11:06:28.009423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 11:06:28.009423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 11:06:28.009423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 11:06:28.009423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 11:06:28.009423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 11:06:28.009423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 11:06:28.009423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 11:06:28.010216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 11:06:28.010216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:06:28.010216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception Sep 19 11:06:28.010216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:06:28.010216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:06:28.010216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:06:28.010216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:06:28.010216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:06:28.010216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:06:28.010216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:06:28.010216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:06:28.010216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:06:28.010216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:06:28.010216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:06:28.010216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application result.read() Sep 19 11:06:28.010216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:06:28.010216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:06:28.011091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:06:28.011091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:06:28.011091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:06:28.011091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:06:28.011091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:06:28.011091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:06:28.011091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 11:06:28.011091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 19 11:06:28.011091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [parameters: {'id': 'c0ec220496ce4aa984fc1b78b718ed85', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 19 11:06:28.011091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 11:06:28.011091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:28.011091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:28.011091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:28.011091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:28.011091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 19 11:06:28.014430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 19 11:06:28.014430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 11:06:28.014430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 11:06:28.014430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 19 11:06:28.014430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:28.014430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:28.014430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:28.014430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:28.014430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:28.014430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:28.014430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:28.014430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:28.014430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:28.014430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:28.014430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:28.014430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:28.014430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:28.016229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:28.016229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 19 11:06:28.016229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 19 11:06:28.016229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:28.016229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:28.016229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 19 11:06:28.016229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 19 11:06:28.016229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c0ec220496ce4aa984fc1b78b718ed85. Sep 19 11:06:28.016229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:28.016229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 467/937] 10.222.0.78 () {60 vars in 1140 bytes} [Mon Sep 19 11:06:27 2022] POST /identity/v3/projects => generated 262 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:28.016229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d20f1-82b5-4d70-a7a6-e1cbf8210cc9 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:28.087855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c10fc-a5bb-4317-ae95-24799e9c8e0c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:28.105373 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:28.116840 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:28.121140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d20f1-82b5-4d70-a7a6-e1cbf8210cc9 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:28.138240 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4c986-feab-40bd-b1a8-f2b20895b72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:28.138878 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4c986-feab-40bd-b1a8-f2b20895b72c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:28.139158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4c986-feab-40bd-b1a8-f2b20895b72c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:28.139445 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4c986-feab-40bd-b1a8-f2b20895b72c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:28.140033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4c986-feab-40bd-b1a8-f2b20895b72c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:28.141417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4c986-feab-40bd-b1a8-f2b20895b72c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:28.150530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-b4a4c986-feab-40bd-b1a8-f2b20895b72c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:06:28.150530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:28.150530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:28.150530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:28.150530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:28.150530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:28.150530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:28.150530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:28.150530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:06:28.150530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:28.150530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:28.150530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:28.150530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:28.150530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:28.150530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:28.150530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:28.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:28.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:28.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:28.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:28.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:28.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:28.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:28.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:28.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:28.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:28.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:28.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:28.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:28.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:28.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:28.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:28.152025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:28.152918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:28.152918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:28.152918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:28.152918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:28.152918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:28.152918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:28.152918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:28.152918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:28.152918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:28.152918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:28.152918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:28.152918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:28.152918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:06:28.152918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:28.152918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 471/938] 10.222.0.78 () {56 vars in 1132 bytes} [Mon Sep 19 11:06:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:28.153734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4c986-feab-40bd-b1a8-f2b20895b72c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:28.163605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bed82b-6160-4c26-b505-4166ad2b5428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:28.164241 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bed82b-6160-4c26-b505-4166ad2b5428 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:28.164521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bed82b-6160-4c26-b505-4166ad2b5428 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:28.164793 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bed82b-6160-4c26-b505-4166ad2b5428 None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:28.165600 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bed82b-6160-4c26-b505-4166ad2b5428 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:28.165909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bed82b-6160-4c26-b505-4166ad2b5428 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:28.167579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bed82b-6160-4c26-b505-4166ad2b5428 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:28.193457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 468/939] 10.222.0.78 () {56 vars in 1149 bytes} [Mon Sep 19 11:06:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:28.193457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bed82b-6160-4c26-b505-4166ad2b5428 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:28.267324 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4c986-feab-40bd-b1a8-f2b20895b72c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:28.296145 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bed82b-6160-4c26-b505-4166ad2b5428 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:28.307063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a748bb-40e1-4b52-82fd-2c995e9f3ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:28.308039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a748bb-40e1-4b52-82fd-2c995e9f3ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:28.308569 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a748bb-40e1-4b52-82fd-2c995e9f3ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:28.309171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a748bb-40e1-4b52-82fd-2c995e9f3ac4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:28.317643 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a748bb-40e1-4b52-82fd-2c995e9f3ac4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:28.319334 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a748bb-40e1-4b52-82fd-2c995e9f3ac4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:28.331199 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46f54f-72c2-4525-bdb6-83dd6290c05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:28.331809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46f54f-72c2-4525-bdb6-83dd6290c05c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:28.332129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46f54f-72c2-4525-bdb6-83dd6290c05c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:28.332360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-49a748bb-40e1-4b52-82fd-2c995e9f3ac4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:28.332360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:28.332360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:28.332360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:28.332360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:28.332360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:28.332360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:28.332360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:28.332360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:28.332360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:28.332360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:28.332360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:28.332360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:06:28.332360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:06:28.332360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:06:28.333198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:06:28.333198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:28.333198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:28.333198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:28.333198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:28.333198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:28.333198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:28.333198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:28.333198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:28.333198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:28.333198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:28.333198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:28.333198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:28.333198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:28.333198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:28.333198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:28.333198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:28.334102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:06:28.334102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:06:28.334102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:06:28.334102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:06:28.334102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:06:28.334102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:06:28.334102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:28.334102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:28.335028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46f54f-72c2-4525-bdb6-83dd6290c05c None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:28.335028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46f54f-72c2-4525-bdb6-83dd6290c05c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:28.335028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46f54f-72c2-4525-bdb6-83dd6290c05c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:28.335269 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 472/940] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:28.335269 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a748bb-40e1-4b52-82fd-2c995e9f3ac4 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:28.335395 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46f54f-72c2-4525-bdb6-83dd6290c05c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:28.348786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 469/941] 10.222.0.78 () {56 vars in 1126 bytes} [Mon Sep 19 11:06:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:28.348786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46f54f-72c2-4525-bdb6-83dd6290c05c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:28.370565 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:28.385777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:28.448477 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a748bb-40e1-4b52-82fd-2c995e9f3ac4 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:28.462455 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46f54f-72c2-4525-bdb6-83dd6290c05c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:28.486991 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5face9-72f5-401f-b36a-a5b5ae453a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:28.487710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5face9-72f5-401f-b36a-a5b5ae453a81 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:28.487992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5face9-72f5-401f-b36a-a5b5ae453a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:28.488268 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5face9-72f5-401f-b36a-a5b5ae453a81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:28.499719 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5face9-72f5-401f-b36a-a5b5ae453a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:28.499955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70801692-caa6-4040-bc08-0c0fff19554f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:28.500954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70801692-caa6-4040-bc08-0c0fff19554f None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:28.501339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70801692-caa6-4040-bc08-0c0fff19554f None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:28.501430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5face9-72f5-401f-b36a-a5b5ae453a81 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:28.501804 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70801692-caa6-4040-bc08-0c0fff19554f None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:28.502465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70801692-caa6-4040-bc08-0c0fff19554f None admin] RBAC: Adding query filter params (name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:28.502769 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70801692-caa6-4040-bc08-0c0fff19554f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:28.504218 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70801692-caa6-4040-bc08-0c0fff19554f None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:28.516277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-9c5face9-72f5-401f-b36a-a5b5ae453a81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:28.516277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:28.516277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:28.516277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:28.516277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:28.516277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:28.516277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:28.516277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:28.516277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:28.516277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:28.516277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:28.516277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:28.516277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:28.516277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:28.516277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:28.516277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:28.517266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:28.517266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:28.517266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:28.517266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:28.517266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:28.517266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:28.517266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:28.517266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:28.517266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:28.517266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:28.517266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:28.517266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:28.517266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:28.517266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:28.517266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:28.517266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:28.517266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:28.518146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:28.518146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:28.518146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:28.522864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 473/942] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:28.531300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 470/943] 10.222.0.78 () {56 vars in 1112 bytes} [Mon Sep 19 11:06:28 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:28.531540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5face9-72f5-401f-b36a-a5b5ae453a81 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:28.531636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70801692-caa6-4040-bc08-0c0fff19554f None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:28.647029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:28.665796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:28.673073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5face9-72f5-401f-b36a-a5b5ae453a81 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:28.678728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70801692-caa6-4040-bc08-0c0fff19554f None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:28.713979 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71f90d-f8ce-4470-8abb-1fbd48e6af82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:28.714845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71f90d-f8ce-4470-8abb-1fbd48e6af82 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:28.715211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71f90d-f8ce-4470-8abb-1fbd48e6af82 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:28.715593 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71f90d-f8ce-4470-8abb-1fbd48e6af82 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:28.716248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71f90d-f8ce-4470-8abb-1fbd48e6af82 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:28.716583 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71f90d-f8ce-4470-8abb-1fbd48e6af82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:28.717952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71f90d-f8ce-4470-8abb-1fbd48e6af82 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:28.723465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 474/944] 10.222.0.78 () {56 vars in 1126 bytes} [Mon Sep 19 11:06:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:28.724708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b6fc0-8e16-4301-a0a3-0f38fe204b13 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:28.725022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b6fc0-8e16-4301-a0a3-0f38fe204b13 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:28.725340 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b6fc0-8e16-4301-a0a3-0f38fe204b13 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:28.726537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 475/945] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:28.728707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9c073-3e1d-4007-bfcf-1e0aaddc675d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:28.729540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b6fc0-8e16-4301-a0a3-0f38fe204b13 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:28.730392 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9c073-3e1d-4007-bfcf-1e0aaddc675d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:28.731116 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9c073-3e1d-4007-bfcf-1e0aaddc675d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:28.731417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9c073-3e1d-4007-bfcf-1e0aaddc675d None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:28.732075 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b9c073-3e1d-4007-bfcf-1e0aaddc675d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:28.732380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b9c073-3e1d-4007-bfcf-1e0aaddc675d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:28.733763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b9c073-3e1d-4007-bfcf-1e0aaddc675d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:28.746762 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 471/946] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:28.748286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9df0a-d63e-4d1b-bca0-86529a0c317f None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:28.748672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9df0a-d63e-4d1b-bca0-86529a0c317f None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:28.749043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9df0a-d63e-4d1b-bca0-86529a0c317f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:28.769457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-f9f9df0a-d63e-4d1b-bca0-86529a0c317f None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:28.771041 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f9f9df0a-d63e-4d1b-bca0-86529a0c317f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:28.856136 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 472/947] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:28.856989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9df0a-d63e-4d1b-bca0-86529a0c317f None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:28.871970 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b6fc0-8e16-4301-a0a3-0f38fe204b13 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:28.922051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544e686-1387-4711-9e6c-fb1b4dace265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:28.923406 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544e686-1387-4711-9e6c-fb1b4dace265 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:28.923701 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544e686-1387-4711-9e6c-fb1b4dace265 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:28.923971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544e686-1387-4711-9e6c-fb1b4dace265 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:28.924647 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544e686-1387-4711-9e6c-fb1b4dace265 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:28.926007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544e686-1387-4711-9e6c-fb1b4dace265 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:28.939284 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:28.970497 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9df0a-d63e-4d1b-bca0-86529a0c317f None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:29.007637 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 476/948] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:06:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:06:29.017987 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4fa72-c662-4b48-afbc-ff0cdbd2ebbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:29.017987 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4fa72-c662-4b48-afbc-ff0cdbd2ebbd None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:29.017987 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4fa72-c662-4b48-afbc-ff0cdbd2ebbd None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:29.017987 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4fa72-c662-4b48-afbc-ff0cdbd2ebbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:29.023627 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4fa72-c662-4b48-afbc-ff0cdbd2ebbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:29.025171 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4fa72-c662-4b48-afbc-ff0cdbd2ebbd None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:29.039326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-cfc4fa72-c662-4b48-afbc-ff0cdbd2ebbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:29.039326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:29.039326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:29.039326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:29.039326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:29.039326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:29.039326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:29.039326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:29.039326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:29.039326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:29.039326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:29.039326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:29.039326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:06:29.039326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:06:29.039326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:06:29.040511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:06:29.040511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:29.040511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:29.040511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:29.040511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:29.040511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:29.040511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:29.040511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:29.040511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:29.040511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:29.040511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:29.040511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:29.040511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:29.040511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:29.040511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:29.040511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:29.040511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:29.041643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:06:29.041643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:06:29.041643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:06:29.041643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:06:29.041643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:06:29.041643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:06:29.041643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:29.041643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:29.041643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 473/949] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:29.048234 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544e686-1387-4711-9e6c-fb1b4dace265 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:29.070411 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:29.138656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4fa72-c662-4b48-afbc-ff0cdbd2ebbd None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:29.187115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544e686-1387-4711-9e6c-fb1b4dace265 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:29.228125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37045e27-c4ab-4bbd-8b0a-f34de4c61437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:29.228772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37045e27-c4ab-4bbd-8b0a-f34de4c61437 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:29.229091 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37045e27-c4ab-4bbd-8b0a-f34de4c61437 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:29.229402 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37045e27-c4ab-4bbd-8b0a-f34de4c61437 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:29.230081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37045e27-c4ab-4bbd-8b0a-f34de4c61437 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:29.230454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37045e27-c4ab-4bbd-8b0a-f34de4c61437 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:29.231936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37045e27-c4ab-4bbd-8b0a-f34de4c61437 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:29.245891 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 477/950] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:06:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:29.252555 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37045e27-c4ab-4bbd-8b0a-f34de4c61437 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:29.292947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4fa72-c662-4b48-afbc-ff0cdbd2ebbd None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:29.326913 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:29.331959 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489e63e-a8a7-4d79-bed0-3f3380f1d2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:29.332886 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489e63e-a8a7-4d79-bed0-3f3380f1d2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:29.333521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489e63e-a8a7-4d79-bed0-3f3380f1d2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:29.334109 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489e63e-a8a7-4d79-bed0-3f3380f1d2c8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:29.335028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489e63e-a8a7-4d79-bed0-3f3380f1d2c8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:29.336716 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489e63e-a8a7-4d79-bed0-3f3380f1d2c8 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:29.340428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:29.347636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-9489e63e-a8a7-4d79-bed0-3f3380f1d2c8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:06:29.347636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:29.347636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:29.347636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:29.347636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:29.347636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:29.347636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:29.347636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:29.347636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:06:29.347636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:29.347636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:29.347636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:29.347636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:29.347636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:29.347636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:29.347636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:29.347636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:29.353619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:29.353619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:29.353619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:29.353619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:29.353619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:29.353619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:29.353619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:29.353619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:29.353619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:29.353619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:29.353619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:29.353619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:29.353619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:29.353619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:29.353619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:29.353619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:29.354532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:29.354532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:29.354532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:29.354532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:29.354532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:29.354532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:29.354532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:29.354532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:29.354532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:29.354532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:29.354532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:29.354532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:29.354532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:06:29.354532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:29.354532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 474/951] 10.222.0.78 () {56 vars in 1126 bytes} [Mon Sep 19 11:06:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:29.357386 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489e63e-a8a7-4d79-bed0-3f3380f1d2c8 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:29.371800 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37045e27-c4ab-4bbd-8b0a-f34de4c61437 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:29.410144 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ff34a-ca88-4a5f-83ee-f0b0cf2f53f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:29.410818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ff34a-ca88-4a5f-83ee-f0b0cf2f53f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:29.411103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ff34a-ca88-4a5f-83ee-f0b0cf2f53f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:29.411424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ff34a-ca88-4a5f-83ee-f0b0cf2f53f7 None admin] PATH_INFO: `/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/5a667fa07b1148998959835d9995ad60/roles/985ecf1b99cb4533a39fad056b155abb` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:29.413879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ff34a-ca88-4a5f-83ee-f0b0cf2f53f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8563e86b7c94ff29e39fb20a7969551, user_id=5a667fa07b1148998959835d9995ad60, role_id=985ecf1b99cb4533a39fad056b155abb)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:29.415356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ff34a-ca88-4a5f-83ee-f0b0cf2f53f7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:29.479452 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489e63e-a8a7-4d79-bed0-3f3380f1d2c8 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:29.486802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 478/952] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:06:29 2022] PUT /identity/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/5a667fa07b1148998959835d9995ad60/roles/985ecf1b99cb4533a39fad056b155abb => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:06:29.528897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d5680-00d2-464b-a93a-9888de4fe2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:29.529903 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d5680-00d2-464b-a93a-9888de4fe2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:29.530339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d5680-00d2-464b-a93a-9888de4fe2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:29.530776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d5680-00d2-464b-a93a-9888de4fe2a1 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:29.531696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d5680-00d2-464b-a93a-9888de4fe2a1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:29.532238 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d5680-00d2-464b-a93a-9888de4fe2a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:29.534077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d5680-00d2-464b-a93a-9888de4fe2a1 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:29.540814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 475/953] 10.222.0.78 () {56 vars in 1122 bytes} [Mon Sep 19 11:06:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:29.547760 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ff34a-ca88-4a5f-83ee-f0b0cf2f53f7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:29.657105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ff34a-ca88-4a5f-83ee-f0b0cf2f53f7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:29.695309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772cbdd-580a-473f-b3bf-e3e1e4d2db80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:29.695931 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772cbdd-580a-473f-b3bf-e3e1e4d2db80 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:29.696215 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772cbdd-580a-473f-b3bf-e3e1e4d2db80 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:29.696487 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772cbdd-580a-473f-b3bf-e3e1e4d2db80 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:29.697119 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772cbdd-580a-473f-b3bf-e3e1e4d2db80 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:29.697419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772cbdd-580a-473f-b3bf-e3e1e4d2db80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:29.698785 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772cbdd-580a-473f-b3bf-e3e1e4d2db80 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:29.700306 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:29.704795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 479/954] 10.222.0.78 () {56 vars in 1122 bytes} [Mon Sep 19 11:06:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:29.710572 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d5680-00d2-464b-a93a-9888de4fe2a1 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:29.773746 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:29.821793 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d5680-00d2-464b-a93a-9888de4fe2a1 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:29.858490 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a453cb-6f73-4211-951f-496e28461fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:29.859398 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a453cb-6f73-4211-951f-496e28461fc4 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:29.859831 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a453cb-6f73-4211-951f-496e28461fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:29.860254 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a453cb-6f73-4211-951f-496e28461fc4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:29.861111 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a453cb-6f73-4211-951f-496e28461fc4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:29.862703 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a453cb-6f73-4211-951f-496e28461fc4 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:29.937708 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 476/955] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:06:29 2022] POST /identity/v3/endpoints => generated 349 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:06:30.815889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fc04c-30c0-4045-be28-224a3c91cfad None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:30.816446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fc04c-30c0-4045-be28-224a3c91cfad None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:30.816446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fc04c-30c0-4045-be28-224a3c91cfad None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:30.818030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 480/956] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:30.824139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfffefa-d33d-4779-a8fa-38d49d18ff47 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:30.824580 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfffefa-d33d-4779-a8fa-38d49d18ff47 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:30.824761 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfffefa-d33d-4779-a8fa-38d49d18ff47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:30.844068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-dbfffefa-d33d-4779-a8fa-38d49d18ff47 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:30.845253 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-dbfffefa-d33d-4779-a8fa-38d49d18ff47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:30.927231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 477/957] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:06:31.191592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fc04c-30c0-4045-be28-224a3c91cfad None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:31.307815 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fc04c-30c0-4045-be28-224a3c91cfad None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:31.344733 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef923cf5-d94e-4d48-973e-4cd50c41b06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:31.345702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef923cf5-d94e-4d48-973e-4cd50c41b06a None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:31.346193 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef923cf5-d94e-4d48-973e-4cd50c41b06a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:31.346671 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef923cf5-d94e-4d48-973e-4cd50c41b06a None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:31.347586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef923cf5-d94e-4d48-973e-4cd50c41b06a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:31.349225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef923cf5-d94e-4d48-973e-4cd50c41b06a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:31.352150 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: WARNING keystone.server.flask.common [None req-ef923cf5-d94e-4d48-973e-4cd50c41b06a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 19 11:06:31.396461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591ca9b-a93c-40c6-b35c-9d130e00032b None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:31.396777 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591ca9b-a93c-40c6-b35c-9d130e00032b None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:31.396777 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591ca9b-a93c-40c6-b35c-9d130e00032b None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:31.399154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.sql.core [None req-ef923cf5-d94e-4d48-973e-4cd50c41b06a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 11:06:31.399154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 19 11:06:31.399154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [parameters: {'user_id': '81fe6f7a43184009a81066ec35ca889e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 19 11:06:31.399154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81737) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 11:06:31.399585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 478/958] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:06:31.404946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170ab57-6f55-4518-b2c0-9fb6f238df9a None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:31.405226 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170ab57-6f55-4518-b2c0-9fb6f238df9a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:31.405501 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170ab57-6f55-4518-b2c0-9fb6f238df9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:31.422242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-ef923cf5-d94e-4d48-973e-4cd50c41b06a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 19 11:06:31.422242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:31.422242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:06:31.422242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:06:31.422242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:06:31.422242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:06:31.422242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:06:31.422242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:06:31.422242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:06:31.422242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:06:31.422242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:06:31.422242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:06:31.422242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:06:31.422242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application result.read() Sep 19 11:06:31.422242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:06:31.423238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:06:31.424202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 11:06:31.424202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application yield resource Sep 19 11:06:31.424202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:06:31.424202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:06:31.424202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 11:06:31.424202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 11:06:31.424202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 11:06:31.424202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 11:06:31.424202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 11:06:31.424202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise self.value Sep 19 11:06:31.424202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 11:06:31.424202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 11:06:31.424202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 11:06:31.424202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application session.commit() Sep 19 11:06:31.424202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 11:06:31.424202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 11:06:31.424202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 11:06:31.425100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 11:06:31.425100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 11:06:31.425100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self.session.flush() Sep 19 11:06:31.425100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 11:06:31.425100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 11:06:31.425100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 11:06:31.425100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 11:06:31.425100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 11:06:31.425100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application compat.raise_( Sep 19 11:06:31.425100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:06:31.425100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception Sep 19 11:06:31.425100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 11:06:31.425100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 11:06:31.425100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 11:06:31.425100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 11:06:31.425100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 11:06:31.425100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 11:06:31.426065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 11:06:31.426065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 11:06:31.426065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 19 11:06:31.426065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 19 11:06:31.426065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 11:06:31.426065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 11:06:31.426065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 11:06:31.426065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 11:06:31.426065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 11:06:31.426065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 11:06:31.426065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 11:06:31.426065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 11:06:31.426065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 11:06:31.426065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 11:06:31.426065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:06:31.426065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception Sep 19 11:06:31.426956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:06:31.426956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:06:31.426956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:06:31.426956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:06:31.426956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:06:31.426956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:06:31.426956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:06:31.426956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:06:31.426956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:06:31.426956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:06:31.426956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:06:31.426956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application result.read() Sep 19 11:06:31.426956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:06:31.426956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:06:31.426956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:06:31.426956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:06:31.428171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:06:31.428171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:06:31.428171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:06:31.428171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:06:31.428171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 11:06:31.428171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 19 11:06:31.428171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [parameters: {'user_id': '81fe6f7a43184009a81066ec35ca889e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 19 11:06:31.428171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 11:06:31.428171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:31.428171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:31.428171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:31.428171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:31.428171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:31.428171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:31.428171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:31.429152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:31.429152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:31.429152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:31.429152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:31.429152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:31.429152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:31.429152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:31.429152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 19 11:06:31.429152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 19 11:06:31.429152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:31.429152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:31.429152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:06:31.429152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:31.429152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:06:31.429152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:31.429152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 19 11:06:31.429152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 19 11:06:31.430368 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 19 11:06:31.430368 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 19 11:06:31.430368 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 11:06:31.430368 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 11:06:31.430368 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 19 11:06:31.430368 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:31.432231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-b170ab57-6f55-4518-b2c0-9fb6f238df9a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:31.432705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 481/959] 10.222.0.78 () {60 vars in 1131 bytes} [Mon Sep 19 11:06:31 2022] POST /identity/v3/users => generated 160 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:31.433471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b170ab57-6f55-4518-b2c0-9fb6f238df9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:31.438666 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef923cf5-d94e-4d48-973e-4cd50c41b06a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:31.469331 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:31.501910 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:31.515016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 479/960] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:06:31.550385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef923cf5-d94e-4d48-973e-4cd50c41b06a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:31.587662 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370c6af-1ed9-4d05-974d-848db1e7749d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:31.588708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370c6af-1ed9-4d05-974d-848db1e7749d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:31.589249 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370c6af-1ed9-4d05-974d-848db1e7749d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:31.589732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370c6af-1ed9-4d05-974d-848db1e7749d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:31.598275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370c6af-1ed9-4d05-974d-848db1e7749d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:31.600406 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370c6af-1ed9-4d05-974d-848db1e7749d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:31.612993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-8370c6af-1ed9-4d05-974d-848db1e7749d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:06:31.612993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:31.612993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:31.612993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:31.612993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:31.612993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:31.612993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:31.612993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:31.612993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:31.612993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:31.612993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:31.612993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:31.612993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:06:31.612993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:06:31.612993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:06:31.612993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:31.621309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:31.622310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:31.622310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:31.622310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:06:31.622310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:06:31.622310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:06:31.622310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:06:31.622310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:06:31.622310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:06:31.622310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:06:31.622310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:31.622310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 482/961] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:06:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:31.622930 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170ab57-6f55-4518-b2c0-9fb6f238df9a None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:31.746997 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170ab57-6f55-4518-b2c0-9fb6f238df9a None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:31.782019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93d479-8e24-41b5-b7c1-ba4a3b67432a None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:31.782366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93d479-8e24-41b5-b7c1-ba4a3b67432a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:31.782580 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93d479-8e24-41b5-b7c1-ba4a3b67432a None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:31.789672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36490152-2947-4a32-a395-5059a9589b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:31.790566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36490152-2947-4a32-a395-5059a9589b3c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:31.790886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 483/962] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:31.791347 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36490152-2947-4a32-a395-5059a9589b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:31.791830 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36490152-2947-4a32-a395-5059a9589b3c None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:31.792148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493660d-b316-4e6f-8223-a04cf9cf0e84 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:31.792441 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493660d-b316-4e6f-8223-a04cf9cf0e84 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:31.792720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493660d-b316-4e6f-8223-a04cf9cf0e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:31.793407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36490152-2947-4a32-a395-5059a9589b3c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:31.793913 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36490152-2947-4a32-a395-5059a9589b3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:31.795695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36490152-2947-4a32-a395-5059a9589b3c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:31.811440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d493660d-b316-4e6f-8223-a04cf9cf0e84 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:31.812979 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d493660d-b316-4e6f-8223-a04cf9cf0e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:31.839206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 480/963] 10.222.0.78 () {56 vars in 1101 bytes} [Mon Sep 19 11:06:31 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:31.839206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36490152-2947-4a32-a395-5059a9589b3c None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:31.874365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:31.895614 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 484/964] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:31.963619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36490152-2947-4a32-a395-5059a9589b3c None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:32.002616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facae37d-0ad1-44f9-b4d8-7a79ff3c06bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:32.003299 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facae37d-0ad1-44f9-b4d8-7a79ff3c06bc None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:32.003581 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facae37d-0ad1-44f9-b4d8-7a79ff3c06bc None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:32.003856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facae37d-0ad1-44f9-b4d8-7a79ff3c06bc None admin] PATH_INFO: `/v3/services/compute` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:32.004457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facae37d-0ad1-44f9-b4d8-7a79ff3c06bc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:32.005866 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facae37d-0ad1-44f9-b4d8-7a79ff3c06bc None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:32.016288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-facae37d-0ad1-44f9-b4d8-7a79ff3c06bc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:06:32.016288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:32.016288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:32.016288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:32.016288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:32.016288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:32.016288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:32.016288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:32.016288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:06:32.016288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:32.016288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:32.016288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:32.016288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:32.016288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:32.016288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:32.016288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:32.016288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:32.017444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:32.017444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:32.017444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:32.017444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:32.017444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:32.017444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:32.017444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:32.017444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:32.017444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:32.017444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:32.017444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:32.017444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:32.017444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:32.017444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:32.017444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:32.017444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:32.018283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:32.018283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:32.018283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:32.018283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:32.018283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:32.018283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:32.018283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:32.018283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:32.018283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:32.018283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:32.018283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:32.018283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:32.018283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:06:32.018283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:32.018283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 481/965] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:32.024041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493660d-b316-4e6f-8223-a04cf9cf0e84 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:32.083874 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:32.145055 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493660d-b316-4e6f-8223-a04cf9cf0e84 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:32.176394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facae37d-0ad1-44f9-b4d8-7a79ff3c06bc None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:32.182974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f916f-0aab-49f0-b566-9416610b42f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:32.182974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f916f-0aab-49f0-b566-9416610b42f1 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:32.183283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f916f-0aab-49f0-b566-9416610b42f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:32.183537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f916f-0aab-49f0-b566-9416610b42f1 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:32.184190 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f916f-0aab-49f0-b566-9416610b42f1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:32.184496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f916f-0aab-49f0-b566-9416610b42f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:32.185874 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f916f-0aab-49f0-b566-9416610b42f1 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:32.193113 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 485/966] 10.222.0.78 () {56 vars in 1112 bytes} [Mon Sep 19 11:06:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:32.198419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271e7e-b7cb-46c3-86b4-7a64c9cbb1cd None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:32.198742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271e7e-b7cb-46c3-86b4-7a64c9cbb1cd None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:32.199028 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271e7e-b7cb-46c3-86b4-7a64c9cbb1cd None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:32.200393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 486/967] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:06:32.201078 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75271e7e-b7cb-46c3-86b4-7a64c9cbb1cd None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:32.279027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:32.290841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facae37d-0ad1-44f9-b4d8-7a79ff3c06bc None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:32.307183 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75271e7e-b7cb-46c3-86b4-7a64c9cbb1cd None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:32.321844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:32.325824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9810bbb-e30d-4929-8ec7-3a4e7803d0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:32.329636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9810bbb-e30d-4929-8ec7-3a4e7803d0de None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:32.329636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9810bbb-e30d-4929-8ec7-3a4e7803d0de None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:32.329636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9810bbb-e30d-4929-8ec7-3a4e7803d0de None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:32.337527 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9810bbb-e30d-4929-8ec7-3a4e7803d0de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:06:32.342329 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3aa853-83d7-4fcf-9fc2-9cadb4affc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:32.342487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-e9810bbb-e30d-4929-8ec7-3a4e7803d0de None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:06:32.342487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:32.342487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:32.342487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:32.342487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:32.342487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:32.342487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:32.342487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:32.342487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:32.342487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:32.342487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:32.342487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:32.342487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:06:32.342487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:06:32.342487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:06:32.342487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:06:32.343740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:06:32.343740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:06:32.343740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:06:32.343740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:06:32.343740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:32.343740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:32.343740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:32.343740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:32.343740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:32.343740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:32.343740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:32.343740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:32.343740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:32.343740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:32.343740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:32.343740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:32.344887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:32.344887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:32.344887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:32.344887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:32.344887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:06:32.344887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:06:32.344887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:06:32.344887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:06:32.344887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:06:32.344887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:06:32.344887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:06:32.344887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:32.345697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3aa853-83d7-4fcf-9fc2-9cadb4affc36 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:32.345697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3aa853-83d7-4fcf-9fc2-9cadb4affc36 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:32.345697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3aa853-83d7-4fcf-9fc2-9cadb4affc36 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:32.345697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3aa853-83d7-4fcf-9fc2-9cadb4affc36 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:32.345697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3aa853-83d7-4fcf-9fc2-9cadb4affc36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:32.346108 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3aa853-83d7-4fcf-9fc2-9cadb4affc36 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:32.348508 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 482/968] 10.222.0.78 () {56 vars in 1120 bytes} [Mon Sep 19 11:06:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:32.349755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a155f80-c2cd-4c6b-b2aa-0815fdde36b7 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:32.350548 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a155f80-c2cd-4c6b-b2aa-0815fdde36b7 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:32.350819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a155f80-c2cd-4c6b-b2aa-0815fdde36b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:32.367202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 487/969] 10.222.0.78 () {56 vars in 1112 bytes} [Mon Sep 19 11:06:32 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:32.367202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3aa853-83d7-4fcf-9fc2-9cadb4affc36 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:32.370491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-7a155f80-c2cd-4c6b-b2aa-0815fdde36b7 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:32.371783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7a155f80-c2cd-4c6b-b2aa-0815fdde36b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:32.454331 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 483/970] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:32.476211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3aa853-83d7-4fcf-9fc2-9cadb4affc36 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:32.528438 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d53be-9b59-4b42-942a-4d3d98349594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:32.529137 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d53be-9b59-4b42-942a-4d3d98349594 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:32.529367 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d53be-9b59-4b42-942a-4d3d98349594 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:32.529783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d53be-9b59-4b42-942a-4d3d98349594 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:32.530764 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d53be-9b59-4b42-942a-4d3d98349594 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:32.531265 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d53be-9b59-4b42-942a-4d3d98349594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:32.533394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d53be-9b59-4b42-942a-4d3d98349594 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:32.546334 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 488/971] 10.222.0.78 () {56 vars in 1115 bytes} [Mon Sep 19 11:06:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:32.552361 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a155f80-c2cd-4c6b-b2aa-0815fdde36b7 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:32.637189 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:32.659590 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a155f80-c2cd-4c6b-b2aa-0815fdde36b7 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:32.696654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d55010-aa23-4adf-a3a1-024a29c94736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:32.697292 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d55010-aa23-4adf-a3a1-024a29c94736 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:32.697571 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d55010-aa23-4adf-a3a1-024a29c94736 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:32.697843 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d55010-aa23-4adf-a3a1-024a29c94736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:32.708067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d55010-aa23-4adf-a3a1-024a29c94736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:32.710045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d55010-aa23-4adf-a3a1-024a29c94736 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:32.724303 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d53be-9b59-4b42-942a-4d3d98349594 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:32.724859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-f8d55010-aa23-4adf-a3a1-024a29c94736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:32.724859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:32.724859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:32.724859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:32.724859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:32.724859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:32.724859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:32.724859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:32.724859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:32.724859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:32.724859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:32.724859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:32.724859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:32.724859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:32.724859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:32.724859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:32.726637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:32.726637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:32.726637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:32.726637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:32.726637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:32.726637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:32.726637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:32.726637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:32.726637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:32.726637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:32.726637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:32.726637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:32.726637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:32.726637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:32.726637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:32.726637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:32.726637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:32.728876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:32.728876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:32.728876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:32.731041 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 484/972] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:32.731920 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d55010-aa23-4adf-a3a1-024a29c94736 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:32.773119 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:32.833729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d53be-9b59-4b42-942a-4d3d98349594 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:32.839032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d55010-aa23-4adf-a3a1-024a29c94736 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:32.872929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:32.874554 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c5431-eeba-480a-89b0-ee22325afdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:32.875180 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c5431-eeba-480a-89b0-ee22325afdd9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:32.875739 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c5431-eeba-480a-89b0-ee22325afdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:32.876008 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c5431-eeba-480a-89b0-ee22325afdd9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:32.876673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c5431-eeba-480a-89b0-ee22325afdd9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:32.878236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c5431-eeba-480a-89b0-ee22325afdd9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:32.879620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4507a-a909-45c1-b349-b37c1175fce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:32.886295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4507a-a909-45c1-b349-b37c1175fce2 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:32.886295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4507a-a909-45c1-b349-b37c1175fce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:32.886295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4507a-a909-45c1-b349-b37c1175fce2 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:32.886295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4507a-a909-45c1-b349-b37c1175fce2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:32.886295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4507a-a909-45c1-b349-b37c1175fce2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:32.886295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4507a-a909-45c1-b349-b37c1175fce2 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:32.888102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-ab0c5431-eeba-480a-89b0-ee22325afdd9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:06:32.888102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:32.888102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:32.888102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:32.888102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:32.888102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:32.888102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:32.888102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:32.888102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:06:32.888102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:32.888102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:32.888102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:32.888102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:32.888102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:32.888102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:32.888102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:32.888102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:32.889041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:32.889041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:32.889041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:32.889041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:32.889041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:32.889041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:32.889041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:32.889041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:32.889041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:32.889041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:32.889041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:32.889041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:32.889041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:32.889041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:32.889041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:32.889041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:32.889888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:32.889888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:32.889888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:32.889888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:32.889888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:32.889888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:32.889888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:32.889888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:32.889888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:32.889888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:32.889888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:32.889888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:32.889888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:06:32.889888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:32.892027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 489/973] 10.222.0.78 () {56 vars in 1126 bytes} [Mon Sep 19 11:06:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:32.895072 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c5431-eeba-480a-89b0-ee22325afdd9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:32.901815 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 485/974] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:32.902529 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4507a-a909-45c1-b349-b37c1175fce2 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:33.006469 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c5431-eeba-480a-89b0-ee22325afdd9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:33.007101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4507a-a909-45c1-b349-b37c1175fce2 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:33.010694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:33.044701 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c8860-e6fa-41e4-b2fd-caf939ba5806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:33.045428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c8860-e6fa-41e4-b2fd-caf939ba5806 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:33.045772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c8860-e6fa-41e4-b2fd-caf939ba5806 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:33.046151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c8860-e6fa-41e4-b2fd-caf939ba5806 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:33.048926 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f55e14-dcaf-4fac-96af-210010cc6088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:33.049686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f55e14-dcaf-4fac-96af-210010cc6088 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:33.049882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f55e14-dcaf-4fac-96af-210010cc6088 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:33.050088 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f55e14-dcaf-4fac-96af-210010cc6088 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:33.050715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f55e14-dcaf-4fac-96af-210010cc6088 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:33.051016 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f55e14-dcaf-4fac-96af-210010cc6088 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:33.052403 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f55e14-dcaf-4fac-96af-210010cc6088 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:33.054839 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c8860-e6fa-41e4-b2fd-caf939ba5806 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:33.056232 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c8860-e6fa-41e4-b2fd-caf939ba5806 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:33.057461 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 490/975] 10.222.0.78 () {56 vars in 1122 bytes} [Mon Sep 19 11:06:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:33.062766 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f55e14-dcaf-4fac-96af-210010cc6088 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:33.068521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-849c8860-e6fa-41e4-b2fd-caf939ba5806 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:06:33.068521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:33.068521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:33.068521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:33.068521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:33.068521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:33.068521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:33.068521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:33.068521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:33.068521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:33.068521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:33.068521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:33.068521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:06:33.068521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:06:33.068521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:06:33.068521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:33.070016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:33.071625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:33.071625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:33.071625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:06:33.071625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:06:33.071625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:06:33.071625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:06:33.071625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:06:33.071625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:06:33.071625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:06:33.071625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:33.074911 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 486/976] 10.222.0.78 () {56 vars in 1117 bytes} [Mon Sep 19 11:06:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:33.075875 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c8860-e6fa-41e4-b2fd-caf939ba5806 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:33.137765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:33.184644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f55e14-dcaf-4fac-96af-210010cc6088 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:33.188364 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c8860-e6fa-41e4-b2fd-caf939ba5806 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:33.220362 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9aec9-eb87-4b11-a0d8-88a2cd7a4eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:33.221142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9aec9-eb87-4b11-a0d8-88a2cd7a4eed None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:33.221544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9aec9-eb87-4b11-a0d8-88a2cd7a4eed None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:33.221925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9aec9-eb87-4b11-a0d8-88a2cd7a4eed None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:33.222660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9aec9-eb87-4b11-a0d8-88a2cd7a4eed None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:33.223078 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9aec9-eb87-4b11-a0d8-88a2cd7a4eed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:33.224475 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9aec9-eb87-4b11-a0d8-88a2cd7a4eed None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:33.225410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2fae6-7d7c-4c28-8650-d13a5197ff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:33.226300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2fae6-7d7c-4c28-8650-d13a5197ff24 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:33.226770 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2fae6-7d7c-4c28-8650-d13a5197ff24 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:33.227221 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2fae6-7d7c-4c28-8650-d13a5197ff24 None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:33.228083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2fae6-7d7c-4c28-8650-d13a5197ff24 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:33.228529 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2fae6-7d7c-4c28-8650-d13a5197ff24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:33.229432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 491/977] 10.222.0.78 () {56 vars in 1122 bytes} [Mon Sep 19 11:06:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:33.230323 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2fae6-7d7c-4c28-8650-d13a5197ff24 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:33.235343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9aec9-eb87-4b11-a0d8-88a2cd7a4eed None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:33.253907 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 487/978] 10.222.0.78 () {56 vars in 1149 bytes} [Mon Sep 19 11:06:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:33.255068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2fae6-7d7c-4c28-8650-d13a5197ff24 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:33.269765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:33.346136 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9aec9-eb87-4b11-a0d8-88a2cd7a4eed None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:33.365219 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2fae6-7d7c-4c28-8650-d13a5197ff24 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:33.380063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:33.382599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cbed9-3e3c-4791-93ff-58fc7b48d770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:33.383238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cbed9-3e3c-4791-93ff-58fc7b48d770 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:33.383517 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cbed9-3e3c-4791-93ff-58fc7b48d770 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:33.383789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cbed9-3e3c-4791-93ff-58fc7b48d770 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:33.384496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cbed9-3e3c-4791-93ff-58fc7b48d770 None admin] RBAC: Adding query filter params (interface=admin, service_id=435a9b783fa046a49d8f334d745152fb, region_id=RegionOne) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:33.384801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cbed9-3e3c-4791-93ff-58fc7b48d770 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:33.386160 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cbed9-3e3c-4791-93ff-58fc7b48d770 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:33.391886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 492/979] 10.222.0.78 () {56 vars in 1249 bytes} [Mon Sep 19 11:06:33 2022] GET /identity/v3/endpoints?service_id=435a9b783fa046a49d8f334d745152fb&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:33.398076 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cbed9-3e3c-4791-93ff-58fc7b48d770 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:33.409484 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea962a-b8c7-47f3-8ca0-9e6a5b81d7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:33.410356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea962a-b8c7-47f3-8ca0-9e6a5b81d7ed None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:33.410636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea962a-b8c7-47f3-8ca0-9e6a5b81d7ed None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:33.411323 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea962a-b8c7-47f3-8ca0-9e6a5b81d7ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:33.485471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea962a-b8c7-47f3-8ca0-9e6a5b81d7ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:33.487153 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea962a-b8c7-47f3-8ca0-9e6a5b81d7ed None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:33.500938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:33.508099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cbed9-3e3c-4791-93ff-58fc7b48d770 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:33.545813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf528cd-4181-417e-8ca7-c0e159973088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:33.546516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf528cd-4181-417e-8ca7-c0e159973088 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:33.546697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf528cd-4181-417e-8ca7-c0e159973088 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:33.546963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf528cd-4181-417e-8ca7-c0e159973088 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:33.547671 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf528cd-4181-417e-8ca7-c0e159973088 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:33.549002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf528cd-4181-417e-8ca7-c0e159973088 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:33.555804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 493/980] 10.222.0.78 () {56 vars in 1087 bytes} [Mon Sep 19 11:06:33 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:06:33.605064 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 488/981] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:06:33 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:06:33.610979 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf528cd-4181-417e-8ca7-c0e159973088 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:33.721572 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf528cd-4181-417e-8ca7-c0e159973088 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:33.759232 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937102fa-336b-405c-87b6-2fb4ea1bab42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:33.760017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937102fa-336b-405c-87b6-2fb4ea1bab42 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:33.760393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937102fa-336b-405c-87b6-2fb4ea1bab42 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:33.760759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937102fa-336b-405c-87b6-2fb4ea1bab42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:33.771582 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937102fa-336b-405c-87b6-2fb4ea1bab42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:33.773087 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937102fa-336b-405c-87b6-2fb4ea1bab42 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:33.788617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-937102fa-336b-405c-87b6-2fb4ea1bab42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:33.788617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:33.788617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:33.788617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:33.788617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:33.788617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:33.788617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:33.788617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:33.788617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:33.788617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:33.788617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:33.788617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:33.788617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:33.788617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:33.788617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:33.788617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:33.789577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:33.789577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:33.789577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:33.789577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:33.789577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:33.789577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:33.789577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:33.789577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:33.789577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:33.789577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:33.789577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:33.789577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:33.789577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:33.789577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:33.789577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:33.789577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:33.789577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:33.790566 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:33.790566 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:33.790566 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:33.790566 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 494/982] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:33.796381 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea962a-b8c7-47f3-8ca0-9e6a5b81d7ed None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:33.807822 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:33.898623 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea962a-b8c7-47f3-8ca0-9e6a5b81d7ed None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:33.933362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878df8e-a240-40f6-95d1-20ce282370fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:33.933977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878df8e-a240-40f6-95d1-20ce282370fe None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:33.934257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878df8e-a240-40f6-95d1-20ce282370fe None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:33.934529 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878df8e-a240-40f6-95d1-20ce282370fe None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:33.935194 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878df8e-a240-40f6-95d1-20ce282370fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:33.935493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878df8e-a240-40f6-95d1-20ce282370fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:33.936862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878df8e-a240-40f6-95d1-20ce282370fe None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:33.952846 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 489/983] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:33.954159 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937102fa-336b-405c-87b6-2fb4ea1bab42 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:33.969811 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:34.059283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937102fa-336b-405c-87b6-2fb4ea1bab42 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:34.096043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df39e9-713b-4d44-aced-f9a5116b8a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:34.096981 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df39e9-713b-4d44-aced-f9a5116b8a44 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:34.097631 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df39e9-713b-4d44-aced-f9a5116b8a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:34.098140 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df39e9-713b-4d44-aced-f9a5116b8a44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:34.116609 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df39e9-713b-4d44-aced-f9a5116b8a44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:34.118084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df39e9-713b-4d44-aced-f9a5116b8a44 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:34.131199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-b3df39e9-713b-4d44-aced-f9a5116b8a44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:34.131199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:34.131199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:34.131199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:34.131199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:34.131199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:34.131199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:34.131199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:34.131199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:34.131199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:34.131199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:34.131199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:34.131199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:06:34.131199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:06:34.131199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:06:34.132189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:06:34.132189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:34.132189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:34.132189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:34.132189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:34.132189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:34.132189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:34.132189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:34.132189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:34.132189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:34.132189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:34.132189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:34.132189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:34.132189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:34.132189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:34.132189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:34.132189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:34.133216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:06:34.133216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:06:34.133216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:06:34.133216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:06:34.133216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:06:34.133216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:06:34.133216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:34.133216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:34.135383 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 495/984] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:34.138125 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878df8e-a240-40f6-95d1-20ce282370fe None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:34.209776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:34.245962 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878df8e-a240-40f6-95d1-20ce282370fe None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:34.280783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a35033-2a45-458e-aee5-c21b35aa2e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:34.281405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a35033-2a45-458e-aee5-c21b35aa2e3d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:34.281685 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a35033-2a45-458e-aee5-c21b35aa2e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:34.281958 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a35033-2a45-458e-aee5-c21b35aa2e3d None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:34.282652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a35033-2a45-458e-aee5-c21b35aa2e3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:34.282957 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a35033-2a45-458e-aee5-c21b35aa2e3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:34.284423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a35033-2a45-458e-aee5-c21b35aa2e3d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:34.302553 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 490/985] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:06:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:34.304470 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df39e9-713b-4d44-aced-f9a5116b8a44 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:34.375660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:34.406681 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df39e9-713b-4d44-aced-f9a5116b8a44 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:34.443776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83756ef-223b-4d7b-a57d-7eb1cd68410e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:34.444394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83756ef-223b-4d7b-a57d-7eb1cd68410e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:34.444674 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83756ef-223b-4d7b-a57d-7eb1cd68410e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:34.444946 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83756ef-223b-4d7b-a57d-7eb1cd68410e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:34.445677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83756ef-223b-4d7b-a57d-7eb1cd68410e None admin] RBAC: Adding query filter params (role.id=985ecf1b99cb4533a39fad056b155abb, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=5a667fa07b1148998959835d9995ad60) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:34.445985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83756ef-223b-4d7b-a57d-7eb1cd68410e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:34.447457 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83756ef-223b-4d7b-a57d-7eb1cd68410e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:34.459703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 496/986] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:06:34 2022] GET /identity/v3/role_assignments?user.id=5a667fa07b1148998959835d9995ad60&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=985ecf1b99cb4533a39fad056b155abb => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:34.586608 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e7b37-11da-4670-92b0-285e59e5b251 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:34.586944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e7b37-11da-4670-92b0-285e59e5b251 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:34.587163 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e7b37-11da-4670-92b0-285e59e5b251 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:34.588761 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 491/987] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:34.594388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6412f4-6fee-4e27-93bf-dc5871092ea1 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:34.594692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6412f4-6fee-4e27-93bf-dc5871092ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:34.594954 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6412f4-6fee-4e27-93bf-dc5871092ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:34.614040 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-af6412f4-6fee-4e27-93bf-dc5871092ea1 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:34.615390 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-af6412f4-6fee-4e27-93bf-dc5871092ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:34.689955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 497/988] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:34.981272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e7b37-11da-4670-92b0-285e59e5b251 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:35.088981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e7b37-11da-4670-92b0-285e59e5b251 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:35.118373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:35.126493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52868992-bdec-46d1-95b7-2b5b8f377085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:35.127187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52868992-bdec-46d1-95b7-2b5b8f377085 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:35.127485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52868992-bdec-46d1-95b7-2b5b8f377085 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:35.127759 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52868992-bdec-46d1-95b7-2b5b8f377085 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:35.136340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52868992-bdec-46d1-95b7-2b5b8f377085 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:35.136340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52868992-bdec-46d1-95b7-2b5b8f377085 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:35.228549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 492/989] 10.222.0.78 () {60 vars in 1141 bytes} [Mon Sep 19 11:06:34 2022] POST /identity/v3/services => generated 242 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:06:35.781559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d037f-d116-4dcf-9f74-ff5b68407d11 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:35.782105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d037f-d116-4dcf-9f74-ff5b68407d11 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:35.782105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d037f-d116-4dcf-9f74-ff5b68407d11 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:35.783622 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 498/990] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:35.789289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f11dcd-b645-47c9-acd5-e782b6e4c05c None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:35.789638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f11dcd-b645-47c9-acd5-e782b6e4c05c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:35.789917 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f11dcd-b645-47c9-acd5-e782b6e4c05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:35.808571 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-e6f11dcd-b645-47c9-acd5-e782b6e4c05c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:35.809888 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e6f11dcd-b645-47c9-acd5-e782b6e4c05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:35.890897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 493/991] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:36.187910 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d037f-d116-4dcf-9f74-ff5b68407d11 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:36.257539 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:36.293837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d037f-d116-4dcf-9f74-ff5b68407d11 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:36.328115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adba278-1a1c-4dc4-bf6f-5e3df6f6b646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:36.329026 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adba278-1a1c-4dc4-bf6f-5e3df6f6b646 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:36.329308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adba278-1a1c-4dc4-bf6f-5e3df6f6b646 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:36.329581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adba278-1a1c-4dc4-bf6f-5e3df6f6b646 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:36.330160 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adba278-1a1c-4dc4-bf6f-5e3df6f6b646 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:36.331572 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adba278-1a1c-4dc4-bf6f-5e3df6f6b646 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:36.341046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-6adba278-1a1c-4dc4-bf6f-5e3df6f6b646 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:06:36.341046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:36.341046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:36.341046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:36.341046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:36.341046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:36.341046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:36.341046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:36.341046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:06:36.341046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:36.341046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:36.341046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:36.341046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:36.341046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:36.341046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:36.341046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:36.341046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:36.342031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:36.342031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:36.342031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:36.342031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:36.342031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:36.342031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:36.342031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:36.342031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:36.342031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:36.342031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:36.342031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:36.342031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:36.342031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:36.342031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:36.342031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:36.342031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:36.342898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:36.342898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:36.342898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:36.342898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:36.342898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:36.342898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:36.342898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:36.342898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:36.342898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:36.342898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:36.342898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:36.342898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:36.342898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:06:36.342898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:36.342898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 499/992] 10.222.0.78 () {56 vars in 1126 bytes} [Mon Sep 19 11:06:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:36.348047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f11dcd-b645-47c9-acd5-e782b6e4c05c None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:36.452164 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f11dcd-b645-47c9-acd5-e782b6e4c05c None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:36.466959 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:36.487521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b9d2e-7860-4d49-80da-140cd5d3955b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:36.491991 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b9d2e-7860-4d49-80da-140cd5d3955b None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:36.491991 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b9d2e-7860-4d49-80da-140cd5d3955b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:36.491991 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b9d2e-7860-4d49-80da-140cd5d3955b None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:36.491991 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b9d2e-7860-4d49-80da-140cd5d3955b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:36.491991 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b9d2e-7860-4d49-80da-140cd5d3955b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:36.491991 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b9d2e-7860-4d49-80da-140cd5d3955b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:36.496874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 494/993] 10.222.0.78 () {56 vars in 1122 bytes} [Mon Sep 19 11:06:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:36.502839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adba278-1a1c-4dc4-bf6f-5e3df6f6b646 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:36.616036 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adba278-1a1c-4dc4-bf6f-5e3df6f6b646 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:36.645664 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:36.652098 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5629a8-96d8-48d9-b23f-7077cbe4a7d1 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:36.652383 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5629a8-96d8-48d9-b23f-7077cbe4a7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:36.652659 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5629a8-96d8-48d9-b23f-7077cbe4a7d1 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:36.654008 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 495/994] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:36.658658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84629e7-0543-4f8f-adf5-915c5cf76a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:36.659191 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00b1f3-5888-4813-a03f-c2d39efb70a9 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:36.663611 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84629e7-0543-4f8f-adf5-915c5cf76a23 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:36.663781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00b1f3-5888-4813-a03f-c2d39efb70a9 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:36.663862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84629e7-0543-4f8f-adf5-915c5cf76a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:36.663934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00b1f3-5888-4813-a03f-c2d39efb70a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:36.664002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84629e7-0543-4f8f-adf5-915c5cf76a23 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:36.664002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84629e7-0543-4f8f-adf5-915c5cf76a23 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:36.664002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84629e7-0543-4f8f-adf5-915c5cf76a23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:36.664002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84629e7-0543-4f8f-adf5-915c5cf76a23 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:36.667802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 500/995] 10.222.0.78 () {56 vars in 1122 bytes} [Mon Sep 19 11:06:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:36.673112 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84629e7-0543-4f8f-adf5-915c5cf76a23 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:36.679207 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-4a00b1f3-5888-4813-a03f-c2d39efb70a9 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:36.680387 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4a00b1f3-5888-4813-a03f-c2d39efb70a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:36.753291 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 496/996] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:06:36.785648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84629e7-0543-4f8f-adf5-915c5cf76a23 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:36.821334 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fddb53-a4cf-4cbb-9796-e8f6fd340450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:36.821947 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fddb53-a4cf-4cbb-9796-e8f6fd340450 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:36.822232 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fddb53-a4cf-4cbb-9796-e8f6fd340450 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:36.822508 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fddb53-a4cf-4cbb-9796-e8f6fd340450 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:36.823203 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fddb53-a4cf-4cbb-9796-e8f6fd340450 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:36.824559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fddb53-a4cf-4cbb-9796-e8f6fd340450 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:36.899213 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 501/997] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:06:36 2022] POST /identity/v3/endpoints => generated 325 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:06:37.020556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00b1f3-5888-4813-a03f-c2d39efb70a9 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:37.135040 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00b1f3-5888-4813-a03f-c2d39efb70a9 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:37.145087 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:37.175935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac47e9-7f89-4917-8652-95bf92d845f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:37.176606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac47e9-7f89-4917-8652-95bf92d845f1 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:37.176881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac47e9-7f89-4917-8652-95bf92d845f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:37.177145 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac47e9-7f89-4917-8652-95bf92d845f1 None admin] PATH_INFO: `/v3/services/image` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:37.177718 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac47e9-7f89-4917-8652-95bf92d845f1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:37.179735 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac47e9-7f89-4917-8652-95bf92d845f1 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:37.189365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-a1ac47e9-7f89-4917-8652-95bf92d845f1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:06:37.189365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:37.189365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:37.189365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:37.189365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:37.189365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:37.189365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:37.189365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:37.189365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:06:37.189365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:37.189365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:37.189365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:37.189365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:37.189365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:37.189365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:37.189365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:37.189365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:37.197227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:37.197227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:37.197227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:37.197227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:37.197227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:37.197227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:37.197227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:37.197227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:37.197227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:37.197227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:37.197227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:37.197227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:37.197227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:37.197227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:37.197227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:37.197227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:37.198417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:37.198417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:37.198417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:37.198417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:37.198417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:37.198417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:37.198417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:37.198417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:37.198417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:37.198417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:37.198417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:37.198417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:37.198417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:06:37.198417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:37.198417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 497/998] 10.222.0.78 () {56 vars in 1105 bytes} [Mon Sep 19 11:06:37 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:37.199568 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fddb53-a4cf-4cbb-9796-e8f6fd340450 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:37.255702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:37.301719 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fddb53-a4cf-4cbb-9796-e8f6fd340450 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:37.340578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0de06e-856b-44e0-b9e7-302fb273245c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:37.341191 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0de06e-856b-44e0-b9e7-302fb273245c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:37.341467 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0de06e-856b-44e0-b9e7-302fb273245c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:37.341742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0de06e-856b-44e0-b9e7-302fb273245c None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:37.342377 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0de06e-856b-44e0-b9e7-302fb273245c None admin] RBAC: Adding query filter params (name=image) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:37.342672 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0de06e-856b-44e0-b9e7-302fb273245c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:37.344058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0de06e-856b-44e0-b9e7-302fb273245c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:37.349328 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 502/999] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:37.355812 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac47e9-7f89-4917-8652-95bf92d845f1 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:37.485954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac47e9-7f89-4917-8652-95bf92d845f1 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:37.493949 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb567d2c-6f02-4ba3-910a-41160d44fb93 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:37.494233 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb567d2c-6f02-4ba3-910a-41160d44fb93 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:37.494515 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb567d2c-6f02-4ba3-910a-41160d44fb93 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:37.496010 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 503/1000] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:06:37.501089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b18e7a-32e8-4684-8806-befb7fb5e633 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:37.501378 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b18e7a-32e8-4684-8806-befb7fb5e633 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:37.501653 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b18e7a-32e8-4684-8806-befb7fb5e633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:37.519351 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-13b18e7a-32e8-4684-8806-befb7fb5e633 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:37.520433 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a083ba1-5306-439d-9d86-8136d8d5c95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:37.520563 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-13b18e7a-32e8-4684-8806-befb7fb5e633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:37.521043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a083ba1-5306-439d-9d86-8136d8d5c95f None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:37.521316 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a083ba1-5306-439d-9d86-8136d8d5c95f None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:37.522127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a083ba1-5306-439d-9d86-8136d8d5c95f None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:37.522765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a083ba1-5306-439d-9d86-8136d8d5c95f None admin] RBAC: Adding query filter params (type=image) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:37.523201 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a083ba1-5306-439d-9d86-8136d8d5c95f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:37.524560 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a083ba1-5306-439d-9d86-8136d8d5c95f None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:37.531336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 498/1001] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:37 2022] GET /identity/v3/services?type=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:37.598169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 504/1002] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:37 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:06:37.872507 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a083ba1-5306-439d-9d86-8136d8d5c95f None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:37.905910 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:37.978427 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a083ba1-5306-439d-9d86-8136d8d5c95f None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:38.013128 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603be85d-84fd-4415-ba1d-01dc696cd793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:38.013765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603be85d-84fd-4415-ba1d-01dc696cd793 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:38.014109 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603be85d-84fd-4415-ba1d-01dc696cd793 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:38.014313 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603be85d-84fd-4415-ba1d-01dc696cd793 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:38.014942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603be85d-84fd-4415-ba1d-01dc696cd793 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:38.016284 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603be85d-84fd-4415-ba1d-01dc696cd793 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:38.026027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-603be85d-84fd-4415-ba1d-01dc696cd793 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:06:38.026027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:38.026027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:38.026027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:38.026027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:38.026027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:38.026027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:38.026027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:38.026027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:06:38.026027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:38.026027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:38.026027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:38.026027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:38.026027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:38.026027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:38.026027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:38.026027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:38.027543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:38.027543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:38.027543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:38.027543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:38.027543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:38.027543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:38.027543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:38.027543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:38.027543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:38.027543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:38.027543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:38.027543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:38.027543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:38.027543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:38.027543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:38.027543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:38.028666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:38.028666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:38.028666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:38.028666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:38.028666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:38.028666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:38.028666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:38.028666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:38.028666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:38.028666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:38.028666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:38.028666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:38.028666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:06:38.028666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:38.031627 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 499/1003] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:38.033808 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b18e7a-32e8-4684-8806-befb7fb5e633 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:38.092558 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:38.158270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b18e7a-32e8-4684-8806-befb7fb5e633 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:38.193142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0ec46-e005-4c56-8bde-d199f42dc3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:38.193810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0ec46-e005-4c56-8bde-d199f42dc3c5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:38.194087 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0ec46-e005-4c56-8bde-d199f42dc3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:38.194357 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0ec46-e005-4c56-8bde-d199f42dc3c5 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:38.194987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0ec46-e005-4c56-8bde-d199f42dc3c5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:38.195307 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0ec46-e005-4c56-8bde-d199f42dc3c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:38.196661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0ec46-e005-4c56-8bde-d199f42dc3c5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:38.202690 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 505/1004] 10.222.0.78 () {56 vars in 1112 bytes} [Mon Sep 19 11:06:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:38.208650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603be85d-84fd-4415-ba1d-01dc696cd793 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:38.307040 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:38.313875 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603be85d-84fd-4415-ba1d-01dc696cd793 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:38.350490 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767aee2d-da19-4397-9b06-da674af79131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:38.351444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767aee2d-da19-4397-9b06-da674af79131 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:38.351948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767aee2d-da19-4397-9b06-da674af79131 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:38.352383 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767aee2d-da19-4397-9b06-da674af79131 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:38.353161 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767aee2d-da19-4397-9b06-da674af79131 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:38.353790 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767aee2d-da19-4397-9b06-da674af79131 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:38.355321 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767aee2d-da19-4397-9b06-da674af79131 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:38.360841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 500/1005] 10.222.0.78 () {56 vars in 1112 bytes} [Mon Sep 19 11:06:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:38.375821 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0ec46-e005-4c56-8bde-d199f42dc3c5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:38.485812 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0ec46-e005-4c56-8bde-d199f42dc3c5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:38.498547 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:38.520403 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdf201-d6e5-4d25-af85-a7991b374566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:38.521021 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdf201-d6e5-4d25-af85-a7991b374566 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:38.521326 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdf201-d6e5-4d25-af85-a7991b374566 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:38.521681 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdf201-d6e5-4d25-af85-a7991b374566 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:38.522882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcdf201-d6e5-4d25-af85-a7991b374566 None admin] RBAC: Adding query filter params (interface=public, service_id=4c24ebec979f4b2da7839ec79055f5e0, region_id=RegionOne) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:38.523203 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcdf201-d6e5-4d25-af85-a7991b374566 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:38.524580 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcdf201-d6e5-4d25-af85-a7991b374566 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:38.534494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 506/1006] 10.222.0.78 () {56 vars in 1251 bytes} [Mon Sep 19 11:06:38 2022] GET /identity/v3/endpoints?service_id=4c24ebec979f4b2da7839ec79055f5e0&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:38.535835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767aee2d-da19-4397-9b06-da674af79131 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:38.648429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767aee2d-da19-4397-9b06-da674af79131 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:38.686405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d445cf-0893-4eae-8ed1-d4e6f808d7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:38.687028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d445cf-0893-4eae-8ed1-d4e6f808d7cf None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:38.687340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d445cf-0893-4eae-8ed1-d4e6f808d7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:38.687616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d445cf-0893-4eae-8ed1-d4e6f808d7cf None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:38.688226 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d445cf-0893-4eae-8ed1-d4e6f808d7cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:38.689596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d445cf-0893-4eae-8ed1-d4e6f808d7cf None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:38.703194 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 501/1007] 10.222.0.78 () {56 vars in 1087 bytes} [Mon Sep 19 11:06:38 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:06:39.131197 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00484990-cedc-43b0-b0fc-ed4f811d74b5 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:39.131699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00484990-cedc-43b0-b0fc-ed4f811d74b5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:39.131800 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00484990-cedc-43b0-b0fc-ed4f811d74b5 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:39.133262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 507/1008] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:39.138304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f73a1-14d0-4a15-b3eb-27a26efa6bd7 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:39.138591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f73a1-14d0-4a15-b3eb-27a26efa6bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:39.138865 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f73a1-14d0-4a15-b3eb-27a26efa6bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:39.157020 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-1e3f73a1-14d0-4a15-b3eb-27a26efa6bd7 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:39.161918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1e3f73a1-14d0-4a15-b3eb-27a26efa6bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:39.179836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:39.236683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 502/1009] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:39.502307 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00484990-cedc-43b0-b0fc-ed4f811d74b5 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:39.611536 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00484990-cedc-43b0-b0fc-ed4f811d74b5 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:39.648545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1a988-5eec-4e53-beb1-cca5f1e7cefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:39.649185 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a988-5eec-4e53-beb1-cca5f1e7cefd None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:39.649457 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a988-5eec-4e53-beb1-cca5f1e7cefd None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:39.649730 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a988-5eec-4e53-beb1-cca5f1e7cefd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:39.650843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1a988-5eec-4e53-beb1-cca5f1e7cefd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:39.652301 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1a988-5eec-4e53-beb1-cca5f1e7cefd None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:39.654022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 508/1010] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:39.659570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f73a1-14d0-4a15-b3eb-27a26efa6bd7 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:39.723234 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172673c-3876-4aed-825a-698c19e6f76d None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:39.723552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172673c-3876-4aed-825a-698c19e6f76d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:39.723793 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172673c-3876-4aed-825a-698c19e6f76d None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:39.726999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 509/1011] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:06:39.730947 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4443628-279c-4313-80ae-01a379e7dc72 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:39.731256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4443628-279c-4313-80ae-01a379e7dc72 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:39.731532 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4443628-279c-4313-80ae-01a379e7dc72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:39.750538 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d4443628-279c-4313-80ae-01a379e7dc72 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:39.751756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d4443628-279c-4313-80ae-01a379e7dc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:39.759777 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f73a1-14d0-4a15-b3eb-27a26efa6bd7 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:39.794390 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24cd5da-b749-4906-bc16-164c9cc6e881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:39.795001 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cd5da-b749-4906-bc16-164c9cc6e881 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:39.795637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cd5da-b749-4906-bc16-164c9cc6e881 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:39.795637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cd5da-b749-4906-bc16-164c9cc6e881 None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:39.795885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:39.796241 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24cd5da-b749-4906-bc16-164c9cc6e881 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:39.797683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24cd5da-b749-4906-bc16-164c9cc6e881 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:39.824158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 510/1012] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:06:39.891551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 503/1013] 10.222.0.78 () {60 vars in 1141 bytes} [Mon Sep 19 11:06:39 2022] POST /identity/v3/projects => generated 311 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:06:40.105262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4443628-279c-4313-80ae-01a379e7dc72 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:40.222015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4443628-279c-4313-80ae-01a379e7dc72 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:40.257174 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341e737-758e-4641-a528-a92882810066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:40.257790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341e737-758e-4641-a528-a92882810066 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:40.258072 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341e737-758e-4641-a528-a92882810066 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:40.258348 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341e737-758e-4641-a528-a92882810066 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:40.259003 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341e737-758e-4641-a528-a92882810066 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:40.260403 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341e737-758e-4641-a528-a92882810066 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:40.331232 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 511/1014] 10.222.0.78 () {60 vars in 1141 bytes} [Mon Sep 19 11:06:40 2022] POST /identity/v3/services => generated 242 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:06:40.885112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692827ef-5be0-476e-9e1c-8ecf8cc4fb34 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:40.885634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692827ef-5be0-476e-9e1c-8ecf8cc4fb34 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:40.885634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692827ef-5be0-476e-9e1c-8ecf8cc4fb34 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:40.886979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 504/1015] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:40.892424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae20fcb-5329-48ae-8d4b-85dfa511f280 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:40.892708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae20fcb-5329-48ae-8d4b-85dfa511f280 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:40.892980 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae20fcb-5329-48ae-8d4b-85dfa511f280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:40.911993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-dae20fcb-5329-48ae-8d4b-85dfa511f280 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:40.913217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-dae20fcb-5329-48ae-8d4b-85dfa511f280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:40.939958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:40.986030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 512/1016] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:40 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:41.257892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692827ef-5be0-476e-9e1c-8ecf8cc4fb34 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:41.267491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:41.360752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692827ef-5be0-476e-9e1c-8ecf8cc4fb34 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:41.394272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc465a2-617f-4f60-a574-02a20764234b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:41.394895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc465a2-617f-4f60-a574-02a20764234b None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:41.395185 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc465a2-617f-4f60-a574-02a20764234b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:41.395459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc465a2-617f-4f60-a574-02a20764234b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:41.396026 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc465a2-617f-4f60-a574-02a20764234b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:41.397424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc465a2-617f-4f60-a574-02a20764234b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:41.406460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-edc465a2-617f-4f60-a574-02a20764234b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:06:41.406460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:41.406460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:41.406460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:41.406460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:41.406460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:41.406460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:41.406460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:41.406460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:06:41.406460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:41.406460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:41.406460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:41.406460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:41.406460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:41.406460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:41.406460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:41.406460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:41.407578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:41.407578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:41.407578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:41.407578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:41.407578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:41.407578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:41.407578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:41.407578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:41.407578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:41.407578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:41.407578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:41.407578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:41.407578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:41.407578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:41.407578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:41.407578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:41.408457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:41.408457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:41.408457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:41.408457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:41.408457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:41.408457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:41.408457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:41.408457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:41.408457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:41.408457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:41.408457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:41.408457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:41.408457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:06:41.408457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:41.410047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 505/1017] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:41.413969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae20fcb-5329-48ae-8d4b-85dfa511f280 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:41.522319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae20fcb-5329-48ae-8d4b-85dfa511f280 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:41.559272 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c81199-6aa5-48a7-b84c-a4e9b3ea17ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:41.559272 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c81199-6aa5-48a7-b84c-a4e9b3ea17ff None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:41.559272 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c81199-6aa5-48a7-b84c-a4e9b3ea17ff None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:41.559272 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c81199-6aa5-48a7-b84c-a4e9b3ea17ff None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:41.559732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c81199-6aa5-48a7-b84c-a4e9b3ea17ff None admin] RBAC: Adding query filter params (name=compute) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:41.559993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c81199-6aa5-48a7-b84c-a4e9b3ea17ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:41.561300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c81199-6aa5-48a7-b84c-a4e9b3ea17ff None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:41.566528 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 513/1018] 10.222.0.78 () {56 vars in 1112 bytes} [Mon Sep 19 11:06:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:41.572239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc465a2-617f-4f60-a574-02a20764234b None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:41.645734 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:41.676573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc465a2-617f-4f60-a574-02a20764234b None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:41.712503 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524074d-f948-4b4f-a21c-b75adb165c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:41.713112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524074d-f948-4b4f-a21c-b75adb165c4b None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:41.713393 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524074d-f948-4b4f-a21c-b75adb165c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:41.713665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524074d-f948-4b4f-a21c-b75adb165c4b None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:41.714290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524074d-f948-4b4f-a21c-b75adb165c4b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:41.714584 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524074d-f948-4b4f-a21c-b75adb165c4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:41.715987 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524074d-f948-4b4f-a21c-b75adb165c4b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:41.721954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 506/1019] 10.222.0.78 () {56 vars in 1112 bytes} [Mon Sep 19 11:06:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:41.727992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c81199-6aa5-48a7-b84c-a4e9b3ea17ff None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:41.754075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:41.830550 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c81199-6aa5-48a7-b84c-a4e9b3ea17ff None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:41.866625 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03367cf8-2959-4bcb-be93-a6ca9e4ac075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:41.867663 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03367cf8-2959-4bcb-be93-a6ca9e4ac075 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:41.868153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03367cf8-2959-4bcb-be93-a6ca9e4ac075 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:41.868647 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03367cf8-2959-4bcb-be93-a6ca9e4ac075 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:41.869522 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03367cf8-2959-4bcb-be93-a6ca9e4ac075 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:41.871080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03367cf8-2959-4bcb-be93-a6ca9e4ac075 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:41.963202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 514/1020] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:06:41 2022] POST /identity/v3/endpoints => generated 334 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:06:42.126755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddefb0-4459-49bc-91ad-eb49e962f3a3 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:42.127325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddefb0-4459-49bc-91ad-eb49e962f3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:42.127325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddefb0-4459-49bc-91ad-eb49e962f3a3 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:42.128876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 507/1021] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:42.134801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebeb6d-2fef-4178-92ff-26eacbdfbf15 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:42.135101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebeb6d-2fef-4178-92ff-26eacbdfbf15 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:42.135411 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebeb6d-2fef-4178-92ff-26eacbdfbf15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:42.155243 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-55ebeb6d-2fef-4178-92ff-26eacbdfbf15 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:42.156458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-55ebeb6d-2fef-4178-92ff-26eacbdfbf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:42.246322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 515/1022] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:42.517243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ddefb0-4459-49bc-91ad-eb49e962f3a3 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:42.539879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50425c-9d7a-4175-ac1b-fc37a5879edd None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:42.540392 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50425c-9d7a-4175-ac1b-fc37a5879edd None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:42.540489 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50425c-9d7a-4175-ac1b-fc37a5879edd None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:42.542000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 516/1023] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:06:42.548458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7719d4-2030-4133-afcc-7bf5481f6fa8 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:42.548739 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7719d4-2030-4133-afcc-7bf5481f6fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:42.549007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7719d4-2030-4133-afcc-7bf5481f6fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:42.562274 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:42.567950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-dd7719d4-2030-4133-afcc-7bf5481f6fa8 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:42.569722 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-dd7719d4-2030-4133-afcc-7bf5481f6fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:42.628911 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ddefb0-4459-49bc-91ad-eb49e962f3a3 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:42.648048 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 517/1024] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:42.667764 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a1db1-8038-4b4b-8599-f9d0dc833d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:42.668401 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a1db1-8038-4b4b-8599-f9d0dc833d70 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:42.668678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a1db1-8038-4b4b-8599-f9d0dc833d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:42.668948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a1db1-8038-4b4b-8599-f9d0dc833d70 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:42.670062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a1db1-8038-4b4b-8599-f9d0dc833d70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:42.671508 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a1db1-8038-4b4b-8599-f9d0dc833d70 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:42.675200 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 508/1025] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:42.679257 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7719d4-2030-4133-afcc-7bf5481f6fa8 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:42.782503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7719d4-2030-4133-afcc-7bf5481f6fa8 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:42.817431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035c6f7-93c3-4a9c-9f99-a1692b83caa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:42.818041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035c6f7-93c3-4a9c-9f99-a1692b83caa5 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:42.818898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035c6f7-93c3-4a9c-9f99-a1692b83caa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:42.819154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035c6f7-93c3-4a9c-9f99-a1692b83caa5 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:42.819821 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035c6f7-93c3-4a9c-9f99-a1692b83caa5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:42.821233 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035c6f7-93c3-4a9c-9f99-a1692b83caa5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:42.830375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:42.918034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a1db1-8038-4b4b-8599-f9d0dc833d70 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:42.921070 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:42.939207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 518/1026] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:06:42 2022] POST /identity/v3/users => generated 285 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:06:43.039289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a1db1-8038-4b4b-8599-f9d0dc833d70 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:43.075797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43deaa-9165-460f-ad24-dafa45a71731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:43.076453 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43deaa-9165-460f-ad24-dafa45a71731 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:43.076733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43deaa-9165-460f-ad24-dafa45a71731 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:43.077006 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43deaa-9165-460f-ad24-dafa45a71731 None admin] PATH_INFO: `/v3/services/image` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:43.077602 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43deaa-9165-460f-ad24-dafa45a71731 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:43.079049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43deaa-9165-460f-ad24-dafa45a71731 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:43.090198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-1f43deaa-9165-460f-ad24-dafa45a71731 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:06:43.090198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:43.090198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:43.090198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:43.090198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:43.090198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:43.090198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:43.090198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:43.090198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:06:43.090198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:43.090198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:43.090198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:43.090198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:43.090198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:43.090198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:43.090198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:43.090198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:43.091380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:43.091380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:43.091380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:43.091380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:43.091380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:43.091380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:43.091380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:43.091380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:43.091380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:43.091380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:43.091380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:43.091380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:43.091380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:43.091380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:43.091380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:43.091380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:43.092212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:43.092212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:43.092212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:43.092212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:43.092212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:43.092212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:43.092212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:43.092212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:43.092212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:43.092212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:43.092212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:43.092212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:43.092212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:06:43.092212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:43.092212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 509/1027] 10.222.0.78 () {56 vars in 1105 bytes} [Mon Sep 19 11:06:42 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:43.098824 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035c6f7-93c3-4a9c-9f99-a1692b83caa5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:43.218964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035c6f7-93c3-4a9c-9f99-a1692b83caa5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:43.255203 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7d638-d513-44ef-80e7-f9a17a66dffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:43.255857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7d638-d513-44ef-80e7-f9a17a66dffe None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:43.256139 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7d638-d513-44ef-80e7-f9a17a66dffe None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:43.256412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7d638-d513-44ef-80e7-f9a17a66dffe None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:43.257042 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7d638-d513-44ef-80e7-f9a17a66dffe None admin] RBAC: Adding query filter params (name=image) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:43.257343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7d638-d513-44ef-80e7-f9a17a66dffe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:43.258708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7d638-d513-44ef-80e7-f9a17a66dffe None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:43.264445 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 519/1028] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:43.270169 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43deaa-9165-460f-ad24-dafa45a71731 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:43.343545 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:43.379602 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43deaa-9165-460f-ad24-dafa45a71731 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:43.417062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c080ddf-8658-4bc4-b2d7-c60a46da1969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:43.417678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c080ddf-8658-4bc4-b2d7-c60a46da1969 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:43.417956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c080ddf-8658-4bc4-b2d7-c60a46da1969 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:43.418229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c080ddf-8658-4bc4-b2d7-c60a46da1969 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:43.418866 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c080ddf-8658-4bc4-b2d7-c60a46da1969 None admin] RBAC: Adding query filter params (type=image) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:43.419184 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c080ddf-8658-4bc4-b2d7-c60a46da1969 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:43.420585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c080ddf-8658-4bc4-b2d7-c60a46da1969 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:43.427233 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 510/1029] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:43.432641 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7d638-d513-44ef-80e7-f9a17a66dffe None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:43.489378 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:43.534733 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7d638-d513-44ef-80e7-f9a17a66dffe None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:43.569880 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5f779-65c2-4582-a328-507c7f2d4c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:43.570503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5f779-65c2-4582-a328-507c7f2d4c01 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:43.570762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5f779-65c2-4582-a328-507c7f2d4c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:43.571036 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5f779-65c2-4582-a328-507c7f2d4c01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:43.571763 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5f779-65c2-4582-a328-507c7f2d4c01 None admin] RBAC: Adding query filter params (interface=public, service_id=d6f06d8b5f2546f6bded9fc56657f4c0, region_id=RegionOne) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:43.572081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5f779-65c2-4582-a328-507c7f2d4c01 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:43.573444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5f779-65c2-4582-a328-507c7f2d4c01 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:43.583423 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 520/1030] 10.222.0.78 () {56 vars in 1251 bytes} [Mon Sep 19 11:06:43 2022] GET /identity/v3/endpoints?service_id=d6f06d8b5f2546f6bded9fc56657f4c0&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:43.584432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c080ddf-8658-4bc4-b2d7-c60a46da1969 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:43.692506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c080ddf-8658-4bc4-b2d7-c60a46da1969 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:43.722069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:43.729108 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb901a04-f2b3-4709-bc52-2d12c489676e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:43.729732 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb901a04-f2b3-4709-bc52-2d12c489676e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:43.739750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb901a04-f2b3-4709-bc52-2d12c489676e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:43.739750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb901a04-f2b3-4709-bc52-2d12c489676e None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:43.739750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb901a04-f2b3-4709-bc52-2d12c489676e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:43.739750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb901a04-f2b3-4709-bc52-2d12c489676e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:43.739750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 511/1031] 10.222.0.78 () {56 vars in 1087 bytes} [Mon Sep 19 11:06:43 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:06:44.314347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81beb8-98ee-4d1c-bb33-78d7ea156f6d None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:44.314745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81beb8-98ee-4d1c-bb33-78d7ea156f6d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:44.314927 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81beb8-98ee-4d1c-bb33-78d7ea156f6d None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:44.316444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 521/1032] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:44.322117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10331c-cbe3-4c7d-b956-37b9a7316f79 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:44.322580 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10331c-cbe3-4c7d-b956-37b9a7316f79 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:44.322682 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10331c-cbe3-4c7d-b956-37b9a7316f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:44.340919 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-3f10331c-cbe3-4c7d-b956-37b9a7316f79 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:44.342114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-3f10331c-cbe3-4c7d-b956-37b9a7316f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:44.410848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 512/1033] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:44.682319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81beb8-98ee-4d1c-bb33-78d7ea156f6d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:44.786637 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81beb8-98ee-4d1c-bb33-78d7ea156f6d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:44.807141 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:44.821425 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc1030-a6ed-44de-b8c1-9e138a81d4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:44.822132 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc1030-a6ed-44de-b8c1-9e138a81d4b8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:44.822410 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc1030-a6ed-44de-b8c1-9e138a81d4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:44.837387 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc1030-a6ed-44de-b8c1-9e138a81d4b8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:44.837387 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc1030-a6ed-44de-b8c1-9e138a81d4b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:06:44.838599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-0ebc1030-a6ed-44de-b8c1-9e138a81d4b8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:06:44.838599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:44.838599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:44.838599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:44.838599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:44.838599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:44.838599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:44.838599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:44.838599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:44.838599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:44.838599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:44.838599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:44.838599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:06:44.838599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:06:44.838599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:06:44.838599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:06:44.839571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:06:44.839571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:06:44.839571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:06:44.839571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:06:44.839571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:44.839571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:44.839571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:44.839571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:44.839571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:44.839571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:44.839571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:44.839571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:44.839571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:44.839571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:44.839571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:44.839571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:44.840428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:44.840428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:44.840428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:44.840428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:44.840428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:06:44.840428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:06:44.840428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:06:44.840428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:06:44.840428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:06:44.840428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:06:44.840428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:06:44.840428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:44.841683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 522/1034] 10.222.0.78 () {56 vars in 1120 bytes} [Mon Sep 19 11:06:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:44.845745 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10331c-cbe3-4c7d-b956-37b9a7316f79 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:44.952128 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10331c-cbe3-4c7d-b956-37b9a7316f79 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:44.976852 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:44.987865 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aab887-53ef-4788-bc2f-4a3269cdf5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:44.988557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aab887-53ef-4788-bc2f-4a3269cdf5a7 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:44.988844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aab887-53ef-4788-bc2f-4a3269cdf5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:44.989118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aab887-53ef-4788-bc2f-4a3269cdf5a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:44.989791 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aab887-53ef-4788-bc2f-4a3269cdf5a7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:44.990129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aab887-53ef-4788-bc2f-4a3269cdf5a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:44.991596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aab887-53ef-4788-bc2f-4a3269cdf5a7 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:45.005439 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 513/1035] 10.222.0.78 () {56 vars in 1115 bytes} [Mon Sep 19 11:06:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:45.006743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc1030-a6ed-44de-b8c1-9e138a81d4b8 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:45.121980 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc1030-a6ed-44de-b8c1-9e138a81d4b8 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:45.158557 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabda1b3-bc6e-42e1-9b2d-15ac8c3d75aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:45.159210 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabda1b3-bc6e-42e1-9b2d-15ac8c3d75aa None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:45.159495 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabda1b3-bc6e-42e1-9b2d-15ac8c3d75aa None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:45.159767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabda1b3-bc6e-42e1-9b2d-15ac8c3d75aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:45.170208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabda1b3-bc6e-42e1-9b2d-15ac8c3d75aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:45.171703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabda1b3-bc6e-42e1-9b2d-15ac8c3d75aa None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:45.187241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-aabda1b3-bc6e-42e1-9b2d-15ac8c3d75aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:45.187241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:45.187241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:45.187241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:45.187241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:45.187241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:45.187241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:45.187241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:45.187241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:45.187241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:45.187241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:45.187241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:45.187241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:45.187241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:45.187241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:45.187241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:45.188262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:45.188262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:45.188262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:45.188262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:45.188262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:45.188262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:45.188262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:45.188262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:45.188262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:45.188262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:45.188262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:45.188262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:45.188262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:45.188262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:45.188262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:45.188262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:45.188262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:45.189158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:45.189158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:45.189158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:45.189158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 523/1036] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:45.196204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aab887-53ef-4788-bc2f-4a3269cdf5a7 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:45.209944 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac4af4-9e59-43c6-ab31-d950deab4da3 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:45.210187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac4af4-9e59-43c6-ab31-d950deab4da3 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:45.210429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac4af4-9e59-43c6-ab31-d950deab4da3 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:45.211802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 524/1037] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:06:45.217030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7019c-7439-4297-8037-0b0d6e871e44 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:45.217318 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7019c-7439-4297-8037-0b0d6e871e44 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:45.217591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7019c-7439-4297-8037-0b0d6e871e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:45.237079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-9bd7019c-7439-4297-8037-0b0d6e871e44 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:45.238281 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9bd7019c-7439-4297-8037-0b0d6e871e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:45.269502 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:45.305640 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aab887-53ef-4788-bc2f-4a3269cdf5a7 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:45.321259 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 525/1038] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:06:45.341907 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494ad51-9dbc-41d0-9f4e-df89192ef50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:45.342560 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494ad51-9dbc-41d0-9f4e-df89192ef50e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:45.342795 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494ad51-9dbc-41d0-9f4e-df89192ef50e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:45.343067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494ad51-9dbc-41d0-9f4e-df89192ef50e None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:45.343742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494ad51-9dbc-41d0-9f4e-df89192ef50e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:45.344042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494ad51-9dbc-41d0-9f4e-df89192ef50e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:45.345415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494ad51-9dbc-41d0-9f4e-df89192ef50e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:45.358076 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 514/1039] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:45.363788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7019c-7439-4297-8037-0b0d6e871e44 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:45.467845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7019c-7439-4297-8037-0b0d6e871e44 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:45.502989 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c5c7f-268d-4fef-9ba2-6efb94dba5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:45.503667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c5c7f-268d-4fef-9ba2-6efb94dba5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:45.503947 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c5c7f-268d-4fef-9ba2-6efb94dba5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:45.504218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c5c7f-268d-4fef-9ba2-6efb94dba5b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:45.511912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c5c7f-268d-4fef-9ba2-6efb94dba5b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:45.513371 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c5c7f-268d-4fef-9ba2-6efb94dba5b6 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:45.525058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-543c5c7f-268d-4fef-9ba2-6efb94dba5b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:06:45.525058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:45.525058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:45.525058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:45.525058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:45.525058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:45.525058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:45.525058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:45.525058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:45.525058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:45.525058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:45.525058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:45.525058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:06:45.525058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:06:45.525058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:06:45.525058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:45.526046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:45.527005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:45.527005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:45.527005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:06:45.527005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:06:45.527005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:06:45.527005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:06:45.527005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:06:45.527005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:06:45.527005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:06:45.527005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:45.530891 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 526/1040] 10.222.0.78 () {56 vars in 1093 bytes} [Mon Sep 19 11:06:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:45.532178 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494ad51-9dbc-41d0-9f4e-df89192ef50e None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:45.582085 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:45.591510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c5c7f-268d-4fef-9ba2-6efb94dba5b6 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:45.618021 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:45.641006 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494ad51-9dbc-41d0-9f4e-df89192ef50e None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:45.680728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a84961-7631-4635-9f1e-0f561333c0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:45.681337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a84961-7631-4635-9f1e-0f561333c0fb None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:45.681650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a84961-7631-4635-9f1e-0f561333c0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:45.681892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a84961-7631-4635-9f1e-0f561333c0fb None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:45.682627 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a84961-7631-4635-9f1e-0f561333c0fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:45.682936 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a84961-7631-4635-9f1e-0f561333c0fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:45.684409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a84961-7631-4635-9f1e-0f561333c0fb None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:45.699219 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c5c7f-268d-4fef-9ba2-6efb94dba5b6 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:45.704969 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 515/1041] 10.222.0.78 () {56 vars in 1133 bytes} [Mon Sep 19 11:06:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:45.711406 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a84961-7631-4635-9f1e-0f561333c0fb None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:45.738397 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c86198-c9a8-4df3-bca1-91d7006a5b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:45.739129 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c86198-c9a8-4df3-bca1-91d7006a5b24 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:45.739454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c86198-c9a8-4df3-bca1-91d7006a5b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:45.740573 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c86198-c9a8-4df3-bca1-91d7006a5b24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:45.753211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c86198-c9a8-4df3-bca1-91d7006a5b24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:06:45.759868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-79c86198-c9a8-4df3-bca1-91d7006a5b24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:06:45.759868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:45.759868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:45.759868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:45.759868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:45.759868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:45.759868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:45.759868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:45.759868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:45.759868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:45.759868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:45.759868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:45.759868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:06:45.759868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:06:45.759868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:06:45.759868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:06:45.760909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:06:45.760909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:06:45.760909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:06:45.760909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:06:45.760909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:45.760909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:45.760909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:45.760909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:45.760909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:45.760909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:45.760909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:45.760909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:45.760909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:45.760909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:45.760909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:45.760909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:45.761749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:45.761749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:45.761749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:45.761749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:45.761749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:06:45.761749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:06:45.761749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:06:45.761749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:06:45.761749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:06:45.761749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:06:45.761749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:06:45.761749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:45.766485 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 527/1042] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:06:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:06:45.767553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c86198-c9a8-4df3-bca1-91d7006a5b24 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:45.826119 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a84961-7631-4635-9f1e-0f561333c0fb None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:45.828824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:45.858794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:45.863665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4598d-ac5d-40e9-a647-18030c953cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:45.864281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4598d-ac5d-40e9-a647-18030c953cee None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:45.864654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4598d-ac5d-40e9-a647-18030c953cee None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:45.864829 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4598d-ac5d-40e9-a647-18030c953cee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:45.879901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c86198-c9a8-4df3-bca1-91d7006a5b24 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:45.916979 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53eaa7-c747-4f0e-b96e-063c969deaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:45.917598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53eaa7-c747-4f0e-b96e-063c969deaab None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:45.917879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53eaa7-c747-4f0e-b96e-063c969deaab None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:45.918186 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53eaa7-c747-4f0e-b96e-063c969deaab None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:45.918849 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53eaa7-c747-4f0e-b96e-063c969deaab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:45.919148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53eaa7-c747-4f0e-b96e-063c969deaab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:45.920550 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53eaa7-c747-4f0e-b96e-063c969deaab None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:45.929335 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 528/1043] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:06:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:45.930624 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d1b9e-b8b0-48e7-a118-3d7200a64f3a None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:45.930914 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d1b9e-b8b0-48e7-a118-3d7200a64f3a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:45.931247 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d1b9e-b8b0-48e7-a118-3d7200a64f3a None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:45.932534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 529/1044] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:45.934598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d1b9e-b8b0-48e7-a118-3d7200a64f3a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:45.936231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4598d-ac5d-40e9-a647-18030c953cee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:45.937948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4598d-ac5d-40e9-a647-18030c953cee None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:46.037288 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d1b9e-b8b0-48e7-a118-3d7200a64f3a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:46.039233 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 516/1045] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:06:45 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 11:06:46.040502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e450d-5672-437d-9702-64f83a430b28 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:46.040788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e450d-5672-437d-9702-64f83a430b28 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:46.041061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e450d-5672-437d-9702-64f83a430b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:46.048078 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:46.058896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-8a8e450d-5672-437d-9702-64f83a430b28 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:46.060134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-8a8e450d-5672-437d-9702-64f83a430b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:46.071498 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e27d5-3691-4eba-8f8a-80796d6f6de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:46.072139 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e27d5-3691-4eba-8f8a-80796d6f6de2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:46.072425 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e27d5-3691-4eba-8f8a-80796d6f6de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:46.072701 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e27d5-3691-4eba-8f8a-80796d6f6de2 None admin] PATH_INFO: `/v3/users/079271a5f81c4299a730eee474368ada` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:46.087021 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e27d5-3691-4eba-8f8a-80796d6f6de2 None admin] RBAC: Authorizing `identity:get_user(user_id=079271a5f81c4299a730eee474368ada)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:46.088517 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e27d5-3691-4eba-8f8a-80796d6f6de2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:46.090173 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 530/1046] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:06:45 2022] GET /identity/v3/users/079271a5f81c4299a730eee474368ada => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:46.090936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e27d5-3691-4eba-8f8a-80796d6f6de2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:46.093528 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:46.135558 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 517/1047] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:46.136487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e450d-5672-437d-9702-64f83a430b28 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:46.207765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e27d5-3691-4eba-8f8a-80796d6f6de2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:46.247572 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ba158-1eb6-44c3-bc9a-94a4c28e8528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:46.248378 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ba158-1eb6-44c3-bc9a-94a4c28e8528 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:46.248666 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ba158-1eb6-44c3-bc9a-94a4c28e8528 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:46.249275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ba158-1eb6-44c3-bc9a-94a4c28e8528 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:46.254976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e450d-5672-437d-9702-64f83a430b28 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:46.261377 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ba158-1eb6-44c3-bc9a-94a4c28e8528 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:46.262812 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ba158-1eb6-44c3-bc9a-94a4c28e8528 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:46.277442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-aa9ba158-1eb6-44c3-bc9a-94a4c28e8528 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:46.277442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:46.277442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:46.277442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:46.277442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:46.277442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:46.277442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:46.277442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:46.277442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:46.277442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:46.277442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:46.277442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:46.277442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:46.277442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:46.277442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:46.277442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:46.278427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:46.278427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:46.278427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:46.278427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:46.278427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:46.278427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:46.278427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:46.278427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:46.278427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:46.278427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:46.278427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:46.278427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:46.278427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:46.278427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:46.278427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:46.278427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:46.278427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:46.279887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:46.279887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:46.279887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:46.279887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 531/1048] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:46.284325 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ba158-1eb6-44c3-bc9a-94a4c28e8528 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:46.294430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1732b-e350-41ae-91f4-1e5e8d74019b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:46.294983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1732b-e350-41ae-91f4-1e5e8d74019b None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:46.295519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1732b-e350-41ae-91f4-1e5e8d74019b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:46.295519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1732b-e350-41ae-91f4-1e5e8d74019b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:46.298351 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:46.352699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:46.366600 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1732b-e350-41ae-91f4-1e5e8d74019b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:46.368089 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1732b-e350-41ae-91f4-1e5e8d74019b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:46.406755 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ba158-1eb6-44c3-bc9a-94a4c28e8528 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:46.444657 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a9481-72a7-4572-83ff-51c50dfd9fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:46.445269 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a9481-72a7-4572-83ff-51c50dfd9fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:46.445552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a9481-72a7-4572-83ff-51c50dfd9fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:46.445834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a9481-72a7-4572-83ff-51c50dfd9fa6 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:46.446516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a9481-72a7-4572-83ff-51c50dfd9fa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:46.446819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a9481-72a7-4572-83ff-51c50dfd9fa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:46.448230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a9481-72a7-4572-83ff-51c50dfd9fa6 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:46.460915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 532/1049] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:46.464739 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a9481-72a7-4572-83ff-51c50dfd9fa6 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:46.487990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 518/1050] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:06:46 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:06:46.487990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1732b-e350-41ae-91f4-1e5e8d74019b None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:46.519109 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:46.569249 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a9481-72a7-4572-83ff-51c50dfd9fa6 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:46.583682 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:46.585095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1732b-e350-41ae-91f4-1e5e8d74019b None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:46.604185 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5c4ad-2d71-47b4-931a-59333d2cf796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:46.604830 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5c4ad-2d71-47b4-931a-59333d2cf796 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:46.605114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5c4ad-2d71-47b4-931a-59333d2cf796 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:46.605389 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5c4ad-2d71-47b4-931a-59333d2cf796 None admin] PATH_INFO: `/v3/services/image` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:46.605977 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c5c4ad-2d71-47b4-931a-59333d2cf796 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:46.607853 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c5c4ad-2d71-47b4-931a-59333d2cf796 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:46.617720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-d4c5c4ad-2d71-47b4-931a-59333d2cf796 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:06:46.617720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:46.617720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:06:46.617720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:06:46.617720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:06:46.617720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:06:46.617720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:06:46.617720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:46.617720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:06:46.617720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:46.617720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:06:46.617720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:46.617720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:46.617720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:46.617720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:46.617720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:46.617720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:46.618857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:46.618857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:46.618857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:46.618857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:46.618857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:46.618857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:46.618857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:06:46.618857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:06:46.618857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:06:46.618857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:06:46.618857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:46.618857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:46.618857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:46.618857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:46.618857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:46.618857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:46.620103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:46.620103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:46.620103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:46.620103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:46.620103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:46.620103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:46.620103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:46.620103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:46.620103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:46.620103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:46.620103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:06:46.620103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:06:46.620103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:06:46.620103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:46.620103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 533/1051] 10.222.0.78 () {56 vars in 1105 bytes} [Mon Sep 19 11:06:46 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:46.621064 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5c4ad-2d71-47b4-931a-59333d2cf796 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:46.624055 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0ec19-de43-446f-8be4-f22381ff46d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:46.625100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0ec19-de43-446f-8be4-f22381ff46d6 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:46.625401 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0ec19-de43-446f-8be4-f22381ff46d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:46.625673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0ec19-de43-446f-8be4-f22381ff46d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:46.632585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce0ec19-de43-446f-8be4-f22381ff46d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:46.634131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce0ec19-de43-446f-8be4-f22381ff46d6 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:46.645217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-8ce0ec19-de43-446f-8be4-f22381ff46d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:46.645217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:46.645217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:46.645217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:46.645217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:46.645217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:46.645217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:46.645217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:46.645217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:46.645217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:46.645217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:46.645217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:46.645217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:06:46.645217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:06:46.645217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:06:46.646378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:06:46.646378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:46.646378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:46.646378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:46.646378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:46.646378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:46.646378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:46.646378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:46.646378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:46.646378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:46.646378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:46.646378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:46.646378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:46.646378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:46.646378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:46.646378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:46.646378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:46.647500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:06:46.647500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:06:46.647500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:06:46.647500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:06:46.647500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:06:46.647500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:06:46.647500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:46.647500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:46.647500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 519/1052] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:46.647500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0ec19-de43-446f-8be4-f22381ff46d6 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:46.728119 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5c4ad-2d71-47b4-931a-59333d2cf796 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:46.741816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:46.748702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0ec19-de43-446f-8be4-f22381ff46d6 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:46.763946 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b7093-56d0-498e-bcb3-233f98a401a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:46.764580 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b7093-56d0-498e-bcb3-233f98a401a9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:46.764863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b7093-56d0-498e-bcb3-233f98a401a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:46.765141 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b7093-56d0-498e-bcb3-233f98a401a9 None admin] PATH_INFO: `/v3/projects/c632b8ec86394fa9b71d93af3293f05d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:46.766262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b7093-56d0-498e-bcb3-233f98a401a9 None admin] RBAC: Authorizing `identity:get_project(project_id=c632b8ec86394fa9b71d93af3293f05d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:46.767787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b7093-56d0-498e-bcb3-233f98a401a9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:46.769442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 534/1053] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:06:46 2022] GET /identity/v3/projects/c632b8ec86394fa9b71d93af3293f05d => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:46.770214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b7093-56d0-498e-bcb3-233f98a401a9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:46.789674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272595dd-8f53-41a9-a5b8-7b17eeac8d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:46.790506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272595dd-8f53-41a9-a5b8-7b17eeac8d58 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:46.790623 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272595dd-8f53-41a9-a5b8-7b17eeac8d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:46.790807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272595dd-8f53-41a9-a5b8-7b17eeac8d58 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:46.791461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272595dd-8f53-41a9-a5b8-7b17eeac8d58 None admin] RBAC: Adding query filter params (name=image) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:46.791760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272595dd-8f53-41a9-a5b8-7b17eeac8d58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:46.793140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272595dd-8f53-41a9-a5b8-7b17eeac8d58 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:46.797913 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 520/1054] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:46.798680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272595dd-8f53-41a9-a5b8-7b17eeac8d58 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:46.826623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:46.880893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b7093-56d0-498e-bcb3-233f98a401a9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:46.898867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272595dd-8f53-41a9-a5b8-7b17eeac8d58 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:46.913892 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99383297-c36b-4bcf-a925-3e0628706fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:46.914507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99383297-c36b-4bcf-a925-3e0628706fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:46.914790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99383297-c36b-4bcf-a925-3e0628706fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:46.915064 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99383297-c36b-4bcf-a925-3e0628706fa3 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:46.915800 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99383297-c36b-4bcf-a925-3e0628706fa3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:46.916118 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99383297-c36b-4bcf-a925-3e0628706fa3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:46.917558 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99383297-c36b-4bcf-a925-3e0628706fa3 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:46.929909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 535/1055] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:06:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:46.930708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99383297-c36b-4bcf-a925-3e0628706fa3 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:46.933232 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caabc6c-4e2d-4a32-abac-79597b2ed85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:46.933860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caabc6c-4e2d-4a32-abac-79597b2ed85a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:46.934125 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caabc6c-4e2d-4a32-abac-79597b2ed85a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:46.934402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caabc6c-4e2d-4a32-abac-79597b2ed85a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:46.935430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caabc6c-4e2d-4a32-abac-79597b2ed85a None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=c632b8ec86394fa9b71d93af3293f05d, user.id=079271a5f81c4299a730eee474368ada) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:46.935804 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caabc6c-4e2d-4a32-abac-79597b2ed85a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:46.937164 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caabc6c-4e2d-4a32-abac-79597b2ed85a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:46.942772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 521/1056] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:06:46 2022] GET /identity/v3/role_assignments?user.id=079271a5f81c4299a730eee474368ada&scope.project.id=c632b8ec86394fa9b71d93af3293f05d&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:46.943485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caabc6c-4e2d-4a32-abac-79597b2ed85a None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:46.978498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:47.061863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99383297-c36b-4bcf-a925-3e0628706fa3 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:47.064113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caabc6c-4e2d-4a32-abac-79597b2ed85a None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:47.071942 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:47.102377 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a227c-c59b-42ac-ab63-d5f14da858d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:47.102915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a227c-c59b-42ac-ab63-d5f14da858d7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:47.103208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a227c-c59b-42ac-ab63-d5f14da858d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:47.103527 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a227c-c59b-42ac-ab63-d5f14da858d7 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:47.104106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a227c-c59b-42ac-ab63-d5f14da858d7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:47.104409 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a227c-c59b-42ac-ab63-d5f14da858d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:47.104749 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f10b0-b43f-4c3d-905b-19dfbd03e89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:47.105365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f10b0-b43f-4c3d-905b-19dfbd03e89a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:47.105723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f10b0-b43f-4c3d-905b-19dfbd03e89a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:47.105835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a227c-c59b-42ac-ab63-d5f14da858d7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:47.105920 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f10b0-b43f-4c3d-905b-19dfbd03e89a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:47.106632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f10b0-b43f-4c3d-905b-19dfbd03e89a None admin] RBAC: Adding query filter params (role.id=985ecf1b99cb4533a39fad056b155abb, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=aba1968529574fe5b4c5d114c874f428) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:47.106944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f10b0-b43f-4c3d-905b-19dfbd03e89a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:47.108416 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f10b0-b43f-4c3d-905b-19dfbd03e89a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:47.115432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 536/1057] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:47.121111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a227c-c59b-42ac-ab63-d5f14da858d7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:47.121698 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 522/1058] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:06:46 2022] GET /identity/v3/role_assignments?user.id=aba1968529574fe5b4c5d114c874f428&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=985ecf1b99cb4533a39fad056b155abb => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:47.239053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a227c-c59b-42ac-ab63-d5f14da858d7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:47.276366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ff482-7636-437a-a762-d983d796a989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:47.276990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ff482-7636-437a-a762-d983d796a989 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:47.277268 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ff482-7636-437a-a762-d983d796a989 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:47.277536 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ff482-7636-437a-a762-d983d796a989 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:47.278212 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ff482-7636-437a-a762-d983d796a989 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:47.279635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ff482-7636-437a-a762-d983d796a989 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:47.391285 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 537/1059] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:06:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:06:49.125716 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4467d3-6439-4f3a-afd5-4608662f860d None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:49.126195 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4467d3-6439-4f3a-afd5-4608662f860d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:49.126287 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4467d3-6439-4f3a-afd5-4608662f860d None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:49.127637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 523/1060] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:49.133744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153bdca-2b7e-4efc-a350-1100dfe8f37f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:49.134155 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153bdca-2b7e-4efc-a350-1100dfe8f37f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:49.134281 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153bdca-2b7e-4efc-a350-1100dfe8f37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:49.153158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8153bdca-2b7e-4efc-a350-1100dfe8f37f None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:49.154587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8153bdca-2b7e-4efc-a350-1100dfe8f37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:49.180277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:49.236832 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 538/1061] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:06:49.291668 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc2a41-9e66-42f4-8b60-d63770eb752e None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:49.291987 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc2a41-9e66-42f4-8b60-d63770eb752e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:49.292224 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc2a41-9e66-42f4-8b60-d63770eb752e None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:49.293568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 524/1062] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:49.298913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3af6c-5ef5-41fc-9e02-ea09eb334e60 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:49.299228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3af6c-5ef5-41fc-9e02-ea09eb334e60 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:49.299525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3af6c-5ef5-41fc-9e02-ea09eb334e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:49.319089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-9fa3af6c-5ef5-41fc-9e02-ea09eb334e60 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:49.320361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9fa3af6c-5ef5-41fc-9e02-ea09eb334e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:49.388343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 539/1063] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:49.503158 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc2a41-9e66-42f4-8b60-d63770eb752e None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:49.593141 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:49.599831 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7752a0-5a3e-42a9-8193-a0abea7da359 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:49.605832 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7752a0-5a3e-42a9-8193-a0abea7da359 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:49.605832 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7752a0-5a3e-42a9-8193-a0abea7da359 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:49.605832 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 540/1064] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:49.607985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cabbcb-86cb-4e11-a4dd-c4023c312e33 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:49.608460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cabbcb-86cb-4e11-a4dd-c4023c312e33 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:49.608930 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cabbcb-86cb-4e11-a4dd-c4023c312e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:49.609934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc2a41-9e66-42f4-8b60-d63770eb752e None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:49.630793 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a6cabbcb-86cb-4e11-a4dd-c4023c312e33 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:49.632359 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a6cabbcb-86cb-4e11-a4dd-c4023c312e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:49.648394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73146c8a-a4cb-432c-9b4b-0039b7f8532f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:49.648937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73146c8a-a4cb-432c-9b4b-0039b7f8532f None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:49.649215 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73146c8a-a4cb-432c-9b4b-0039b7f8532f None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:49.649488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73146c8a-a4cb-432c-9b4b-0039b7f8532f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:49.664786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73146c8a-a4cb-432c-9b4b-0039b7f8532f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:06:49.670227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-73146c8a-a4cb-432c-9b4b-0039b7f8532f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:06:49.670227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:49.670227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:49.670227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:49.670227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:49.670227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:49.670227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:49.670227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:49.670227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:49.670227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:49.670227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:49.670227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:49.670227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:06:49.670227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:06:49.670227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:06:49.670227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:06:49.671378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:06:49.671378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:06:49.671378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:06:49.671378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:06:49.671378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:49.671378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:49.671378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:49.671378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:49.671378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:49.671378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:49.671378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:49.671378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:49.671378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:49.671378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:49.671378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:49.671378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:49.672295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:49.672295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:49.672295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:49.672295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:49.672295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:06:49.672295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:06:49.672295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:06:49.672295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:06:49.672295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:06:49.672295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:06:49.672295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:06:49.672295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:49.672295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 525/1065] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:06:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:49.673120 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73146c8a-a4cb-432c-9b4b-0039b7f8532f None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:49.715120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 541/1066] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:06:49.717072 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cabbcb-86cb-4e11-a4dd-c4023c312e33 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:49.733417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:49.783596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73146c8a-a4cb-432c-9b4b-0039b7f8532f None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:49.820964 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1d43c-7947-4378-9c16-0ce60d527d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:49.821337 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cabbcb-86cb-4e11-a4dd-c4023c312e33 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:49.821651 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1d43c-7947-4378-9c16-0ce60d527d97 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:49.821933 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1d43c-7947-4378-9c16-0ce60d527d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:49.822204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1d43c-7947-4378-9c16-0ce60d527d97 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:49.832835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1d43c-7947-4378-9c16-0ce60d527d97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:06:49.837721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-e3e1d43c-7947-4378-9c16-0ce60d527d97 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:06:49.837721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:49.837721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:49.837721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:49.837721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:49.837721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:49.837721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:49.837721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:49.837721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:49.837721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:49.837721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:49.837721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:49.837721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:06:49.837721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:06:49.837721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:06:49.837721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:06:49.838670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:06:49.838670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:06:49.838670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:06:49.838670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:06:49.838670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:49.838670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:49.838670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:49.838670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:49.838670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:49.838670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:49.838670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:49.838670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:49.838670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:49.838670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:49.838670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:49.838670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:49.839591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:49.839591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:49.839591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:49.839591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:49.839591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:06:49.839591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:06:49.839591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:06:49.839591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:06:49.839591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:06:49.839591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:06:49.839591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:06:49.839591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:49.839591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 526/1067] 10.222.0.78 () {56 vars in 1120 bytes} [Mon Sep 19 11:06:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:49.844687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1d43c-7947-4378-9c16-0ce60d527d97 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:49.854420 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:49.861051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73522a9b-7992-4d27-9d29-0325fe85f34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:49.861883 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73522a9b-7992-4d27-9d29-0325fe85f34d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:49.862566 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73522a9b-7992-4d27-9d29-0325fe85f34d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:49.862844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73522a9b-7992-4d27-9d29-0325fe85f34d None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:49.863831 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73522a9b-7992-4d27-9d29-0325fe85f34d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:49.864139 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73522a9b-7992-4d27-9d29-0325fe85f34d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:49.869129 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73522a9b-7992-4d27-9d29-0325fe85f34d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:49.880163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 542/1068] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:06:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:49.881761 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73522a9b-7992-4d27-9d29-0325fe85f34d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:49.952733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1d43c-7947-4378-9c16-0ce60d527d97 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:49.963227 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:49.988697 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484d55d-df0c-42b1-81de-c3d20cdca5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:49.989306 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484d55d-df0c-42b1-81de-c3d20cdca5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:49.989584 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484d55d-df0c-42b1-81de-c3d20cdca5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:49.989888 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484d55d-df0c-42b1-81de-c3d20cdca5a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:49.990356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73522a9b-7992-4d27-9d29-0325fe85f34d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:49.990550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484d55d-df0c-42b1-81de-c3d20cdca5a5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:49.990854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484d55d-df0c-42b1-81de-c3d20cdca5a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:49.992265 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484d55d-df0c-42b1-81de-c3d20cdca5a5 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:50.000497 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 527/1069] 10.222.0.78 () {56 vars in 1115 bytes} [Mon Sep 19 11:06:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:50.004284 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484d55d-df0c-42b1-81de-c3d20cdca5a5 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:50.025696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31602e1-1e22-487c-b18c-53acae92601c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:50.026335 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31602e1-1e22-487c-b18c-53acae92601c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:50.026613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31602e1-1e22-487c-b18c-53acae92601c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:50.026885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31602e1-1e22-487c-b18c-53acae92601c None admin] PATH_INFO: `/v3/users/079271a5f81c4299a730eee474368ada` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:50.028085 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31602e1-1e22-487c-b18c-53acae92601c None admin] RBAC: Authorizing `identity:get_user(user_id=079271a5f81c4299a730eee474368ada)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:50.029503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31602e1-1e22-487c-b18c-53acae92601c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:50.031081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 543/1070] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:06:49 2022] GET /identity/v3/users/079271a5f81c4299a730eee474368ada => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:50.031927 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31602e1-1e22-487c-b18c-53acae92601c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:50.074127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:50.113123 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484d55d-df0c-42b1-81de-c3d20cdca5a5 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:50.131719 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31602e1-1e22-487c-b18c-53acae92601c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:50.148521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a436b84-bf40-45cc-b356-cfa2a4b35203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:50.149148 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a436b84-bf40-45cc-b356-cfa2a4b35203 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:50.149404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a436b84-bf40-45cc-b356-cfa2a4b35203 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:50.149678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a436b84-bf40-45cc-b356-cfa2a4b35203 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:50.166739 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db934e-7903-4dc1-ac92-71b0a239792f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:50.167406 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db934e-7903-4dc1-ac92-71b0a239792f None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:50.167736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db934e-7903-4dc1-ac92-71b0a239792f None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:50.168010 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db934e-7903-4dc1-ac92-71b0a239792f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:50.178796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db934e-7903-4dc1-ac92-71b0a239792f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:50.180354 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db934e-7903-4dc1-ac92-71b0a239792f None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:50.182148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:50.194969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-78db934e-7903-4dc1-ac92-71b0a239792f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:50.194969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:50.194969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:50.194969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:50.194969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:50.194969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:50.194969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:50.194969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:50.194969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:50.194969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:50.194969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:50.194969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:50.194969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:50.194969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:50.194969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:50.194969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:50.196184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:50.196184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:50.196184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:50.196184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:50.196184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:50.196184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:50.196184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:50.196184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:50.196184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:50.196184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:50.196184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:50.196184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:50.196184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:50.196184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:50.196184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:50.196184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:50.196184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:50.197101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:50.197101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:50.197101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:50.197101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 544/1071] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:50.197101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db934e-7903-4dc1-ac92-71b0a239792f None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:50.221323 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a436b84-bf40-45cc-b356-cfa2a4b35203 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:50.222695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a436b84-bf40-45cc-b356-cfa2a4b35203 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:50.293953 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:50.307418 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db934e-7903-4dc1-ac92-71b0a239792f None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:50.328434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 528/1072] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:06:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:06:50.329297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a436b84-bf40-45cc-b356-cfa2a4b35203 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:50.343179 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e7515-0836-478b-aa7d-48b9cde295fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:50.343821 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e7515-0836-478b-aa7d-48b9cde295fe None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:50.344098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e7515-0836-478b-aa7d-48b9cde295fe None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:50.344369 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e7515-0836-478b-aa7d-48b9cde295fe None admin] PATH_INFO: `/v3/projects/c632b8ec86394fa9b71d93af3293f05d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:50.345476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e7515-0836-478b-aa7d-48b9cde295fe None admin] RBAC: Authorizing `identity:get_project(project_id=c632b8ec86394fa9b71d93af3293f05d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:50.346915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e7515-0836-478b-aa7d-48b9cde295fe None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:50.348566 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 545/1073] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:06:50 2022] GET /identity/v3/projects/c632b8ec86394fa9b71d93af3293f05d => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:50.349351 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e7515-0836-478b-aa7d-48b9cde295fe None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:50.415309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:50.428171 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a436b84-bf40-45cc-b356-cfa2a4b35203 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:50.453865 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e7515-0836-478b-aa7d-48b9cde295fe None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:50.462415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09095984-07ad-4a15-a574-6d2892f7a8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:50.463023 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09095984-07ad-4a15-a574-6d2892f7a8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:50.463317 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09095984-07ad-4a15-a574-6d2892f7a8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:50.463593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09095984-07ad-4a15-a574-6d2892f7a8b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:50.464230 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09095984-07ad-4a15-a574-6d2892f7a8b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:50.464531 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09095984-07ad-4a15-a574-6d2892f7a8b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:50.465898 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09095984-07ad-4a15-a574-6d2892f7a8b1 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:50.479566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 529/1074] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:50.480364 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09095984-07ad-4a15-a574-6d2892f7a8b1 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:50.492163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4d393-29e3-440e-9b19-3f53abaedd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:50.492805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4d393-29e3-440e-9b19-3f53abaedd45 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:50.493104 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4d393-29e3-440e-9b19-3f53abaedd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:50.493416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4d393-29e3-440e-9b19-3f53abaedd45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:50.504280 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4d393-29e3-440e-9b19-3f53abaedd45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:50.506431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4d393-29e3-440e-9b19-3f53abaedd45 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:50.510184 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:50.521813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-56c4d393-29e3-440e-9b19-3f53abaedd45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:50.521813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:50.521813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:50.521813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:50.521813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:50.521813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:50.521813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:50.521813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:50.521813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:50.521813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:50.521813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:50.521813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:50.521813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:50.521813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:50.521813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:50.521813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:50.522843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:50.522843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:50.522843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:50.522843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:50.522843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:50.522843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:50.522843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:50.522843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:50.522843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:50.522843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:50.522843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:50.522843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:50.522843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:50.522843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:50.522843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:50.522843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:50.522843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:50.523819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:50.523819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:50.523819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:50.523819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 546/1075] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:50.524060 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4d393-29e3-440e-9b19-3f53abaedd45 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:50.587897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09095984-07ad-4a15-a574-6d2892f7a8b1 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:50.621836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd993f3-fbab-437c-b993-78daa7391553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:50.622495 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd993f3-fbab-437c-b993-78daa7391553 None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:50.622768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd993f3-fbab-437c-b993-78daa7391553 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:50.623042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd993f3-fbab-437c-b993-78daa7391553 None admin] PATH_INFO: `/v3/projects/c632b8ec86394fa9b71d93af3293f05d/users/079271a5f81c4299a730eee474368ada/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:50.625232 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd993f3-fbab-437c-b993-78daa7391553 None admin] RBAC: Authorizing `identity:create_grant(project_id=c632b8ec86394fa9b71d93af3293f05d, user_id=079271a5f81c4299a730eee474368ada, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:50.626657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd993f3-fbab-437c-b993-78daa7391553 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:50.641172 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4d393-29e3-440e-9b19-3f53abaedd45 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:50.655083 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:50.675399 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064e5fc-b8cb-439e-8789-d4f2e3f69e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:50.676032 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064e5fc-b8cb-439e-8789-d4f2e3f69e26 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:50.676317 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064e5fc-b8cb-439e-8789-d4f2e3f69e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:50.676589 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064e5fc-b8cb-439e-8789-d4f2e3f69e26 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:50.684457 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9064e5fc-b8cb-439e-8789-d4f2e3f69e26 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:50.685894 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9064e5fc-b8cb-439e-8789-d4f2e3f69e26 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:50.691076 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 530/1076] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:06:50 2022] PUT /identity/v3/projects/c632b8ec86394fa9b71d93af3293f05d/users/079271a5f81c4299a730eee474368ada/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:06:50.691076 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd993f3-fbab-437c-b993-78daa7391553 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:50.698101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-9064e5fc-b8cb-439e-8789-d4f2e3f69e26 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:06:50.698101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:50.698101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:50.698101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:50.698101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:50.698101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:50.698101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:50.698101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:50.698101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:50.698101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:50.698101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:50.698101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:50.698101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:06:50.698101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:06:50.698101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:06:50.698101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:50.699208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:50.700214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:50.700214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:50.700214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:06:50.700214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:06:50.700214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:06:50.700214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:06:50.700214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:06:50.700214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:06:50.700214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:06:50.700214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:50.700214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 547/1077] 10.222.0.78 () {56 vars in 1093 bytes} [Mon Sep 19 11:06:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:50.705603 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064e5fc-b8cb-439e-8789-d4f2e3f69e26 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:50.799366 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:50.812962 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd993f3-fbab-437c-b993-78daa7391553 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:50.828988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064e5fc-b8cb-439e-8789-d4f2e3f69e26 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:50.852532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13628a4-7f0c-49e5-96a4-f1dfc79d25c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:50.853157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13628a4-7f0c-49e5-96a4-f1dfc79d25c6 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:50.853440 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13628a4-7f0c-49e5-96a4-f1dfc79d25c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:50.853717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13628a4-7f0c-49e5-96a4-f1dfc79d25c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:50.854371 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13628a4-7f0c-49e5-96a4-f1dfc79d25c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:50.854685 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13628a4-7f0c-49e5-96a4-f1dfc79d25c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:50.856107 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13628a4-7f0c-49e5-96a4-f1dfc79d25c6 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:50.872286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 531/1078] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:50.878922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c66ad-9258-4fdf-b00a-bb5bd8a42004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:50.879758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c66ad-9258-4fdf-b00a-bb5bd8a42004 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:50.880052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c66ad-9258-4fdf-b00a-bb5bd8a42004 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:50.880946 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c66ad-9258-4fdf-b00a-bb5bd8a42004 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:50.881725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c66ad-9258-4fdf-b00a-bb5bd8a42004 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:50.882026 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c66ad-9258-4fdf-b00a-bb5bd8a42004 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:50.883545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c66ad-9258-4fdf-b00a-bb5bd8a42004 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:50.904493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 548/1079] 10.222.0.78 () {56 vars in 1133 bytes} [Mon Sep 19 11:06:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:50.911005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13628a4-7f0c-49e5-96a4-f1dfc79d25c6 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:51.029783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13628a4-7f0c-49e5-96a4-f1dfc79d25c6 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:51.070258 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130344b-932b-46f6-9229-ccb388aae8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:51.070825 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130344b-932b-46f6-9229-ccb388aae8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:51.071106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130344b-932b-46f6-9229-ccb388aae8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:51.071468 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130344b-932b-46f6-9229-ccb388aae8d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:51.083568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130344b-932b-46f6-9229-ccb388aae8d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:51.085040 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130344b-932b-46f6-9229-ccb388aae8d2 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:51.094915 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:51.102054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-d130344b-932b-46f6-9229-ccb388aae8d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:51.102054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:51.102054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:51.102054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:51.102054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:51.102054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:51.102054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:51.102054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:51.102054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:51.102054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:51.102054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:51.102054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:51.102054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:51.102054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:51.102054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:51.102054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:51.110081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:51.110081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:51.110081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:51.110081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:51.110081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:51.110081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:51.110081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:51.110081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:51.110081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:51.110081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:51.110081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:51.110081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:51.110081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:51.110081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:51.110081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:51.110081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:51.110081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:51.111344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:51.111344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:51.111344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:51.111344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 532/1080] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:51.111591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c66ad-9258-4fdf-b00a-bb5bd8a42004 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:51.149949 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:51.236903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c66ad-9258-4fdf-b00a-bb5bd8a42004 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:51.287781 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f2d3c-6caf-4528-aacf-a9150bbb4414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:51.288784 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f2d3c-6caf-4528-aacf-a9150bbb4414 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:51.289275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f2d3c-6caf-4528-aacf-a9150bbb4414 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:51.289754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f2d3c-6caf-4528-aacf-a9150bbb4414 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:51.290900 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f2d3c-6caf-4528-aacf-a9150bbb4414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:51.291487 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f2d3c-6caf-4528-aacf-a9150bbb4414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:51.294055 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f2d3c-6caf-4528-aacf-a9150bbb4414 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:51.315351 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 549/1081] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:51.320958 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130344b-932b-46f6-9229-ccb388aae8d2 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:51.426033 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130344b-932b-46f6-9229-ccb388aae8d2 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:51.461618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471670d-90fa-401c-bf32-94e3af60f443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:51.462244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471670d-90fa-401c-bf32-94e3af60f443 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:51.462520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471670d-90fa-401c-bf32-94e3af60f443 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:51.462790 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471670d-90fa-401c-bf32-94e3af60f443 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:51.469791 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471670d-90fa-401c-bf32-94e3af60f443 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:51.471269 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471670d-90fa-401c-bf32-94e3af60f443 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:51.483296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-3471670d-90fa-401c-bf32-94e3af60f443 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:51.483296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:51.483296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:51.483296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:51.483296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:51.483296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:51.483296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:51.483296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:51.483296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:51.483296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:51.483296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:51.483296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:51.483296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:06:51.483296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:06:51.483296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:06:51.484290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:06:51.484290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:51.484290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:51.484290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:51.484290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:51.484290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:51.484290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:51.484290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:51.484290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:51.484290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:51.484290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:51.484290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:51.484290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:51.484290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:51.484290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:51.484290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:51.484290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:51.485606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:06:51.485606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:06:51.485606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:06:51.485606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:06:51.485606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:06:51.485606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:06:51.485606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:51.485606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:51.485606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 533/1082] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:51.490193 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f2d3c-6caf-4528-aacf-a9150bbb4414 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:51.581984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:51.590715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f2d3c-6caf-4528-aacf-a9150bbb4414 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:51.626720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fb63a-ba54-403d-95ca-1af9c16a5ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:51.627361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fb63a-ba54-403d-95ca-1af9c16a5ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:51.627642 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fb63a-ba54-403d-95ca-1af9c16a5ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:51.627914 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fb63a-ba54-403d-95ca-1af9c16a5ad8 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:51.628603 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fb63a-ba54-403d-95ca-1af9c16a5ad8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:51.628903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fb63a-ba54-403d-95ca-1af9c16a5ad8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:51.630392 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fb63a-ba54-403d-95ca-1af9c16a5ad8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:51.642269 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 550/1083] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:06:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:51.648239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471670d-90fa-401c-bf32-94e3af60f443 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:51.712723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:51.770985 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471670d-90fa-401c-bf32-94e3af60f443 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:51.804751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eadbc3-5b31-4199-954b-e674250199d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:51.805415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eadbc3-5b31-4199-954b-e674250199d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:51.805693 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eadbc3-5b31-4199-954b-e674250199d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:51.805967 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eadbc3-5b31-4199-954b-e674250199d8 None admin] PATH_INFO: `/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/aba1968529574fe5b4c5d114c874f428/roles/985ecf1b99cb4533a39fad056b155abb` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:51.808111 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eadbc3-5b31-4199-954b-e674250199d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8563e86b7c94ff29e39fb20a7969551, user_id=aba1968529574fe5b4c5d114c874f428, role_id=985ecf1b99cb4533a39fad056b155abb)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:51.809554 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eadbc3-5b31-4199-954b-e674250199d8 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:51.867505 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 534/1084] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:06:51 2022] PUT /identity/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/aba1968529574fe5b4c5d114c874f428/roles/985ecf1b99cb4533a39fad056b155abb => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:06:52.895238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdf6f1-13df-422e-924b-47a745cae43e None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:52.895797 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdf6f1-13df-422e-924b-47a745cae43e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:52.895797 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdf6f1-13df-422e-924b-47a745cae43e None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:52.897272 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 551/1085] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:52.902832 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e6367-9b26-4170-b20a-ed47e990094e None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:52.903295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e6367-9b26-4170-b20a-ed47e990094e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:52.903403 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e6367-9b26-4170-b20a-ed47e990094e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:52.921953 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-852e6367-9b26-4170-b20a-ed47e990094e None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:52.923192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-852e6367-9b26-4170-b20a-ed47e990094e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:52.997215 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 535/1086] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:53.265355 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdf6f1-13df-422e-924b-47a745cae43e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:53.387862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdf6f1-13df-422e-924b-47a745cae43e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:53.440069 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2057fb5-2b54-4e84-a051-77b0b5782176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:53.440069 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2057fb5-2b54-4e84-a051-77b0b5782176 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:53.440069 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2057fb5-2b54-4e84-a051-77b0b5782176 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:53.440069 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2057fb5-2b54-4e84-a051-77b0b5782176 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:53.440904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2057fb5-2b54-4e84-a051-77b0b5782176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:06:53.440904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-c2057fb5-2b54-4e84-a051-77b0b5782176 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:06:53.440904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:53.440904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:53.440904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:53.440904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:53.440904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:53.440904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:53.440904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:53.440904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:53.440904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:53.440904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:53.440904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:53.440904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:06:53.441695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:06:53.441695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:06:53.441695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:06:53.441695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:06:53.441695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:06:53.441695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:06:53.441695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:06:53.441695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:53.441695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:53.441695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:53.441695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:53.441695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:53.441695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:53.441695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:53.441695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:53.441695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:53.441695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:53.442587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:53.442587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:53.442587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:53.442587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:53.442587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:53.442587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:53.442587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:06:53.442587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:06:53.442587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:06:53.442587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:06:53.442587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:06:53.442587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:06:53.442587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:06:53.442587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:53.442587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 552/1087] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:06:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:53.448014 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e6367-9b26-4170-b20a-ed47e990094e None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:53.507365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:53.557609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e6367-9b26-4170-b20a-ed47e990094e None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:53.591850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada7d15-c06d-4dd5-a457-c0a18f2951cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:53.592457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada7d15-c06d-4dd5-a457-c0a18f2951cf None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:53.592736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada7d15-c06d-4dd5-a457-c0a18f2951cf None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:53.593010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada7d15-c06d-4dd5-a457-c0a18f2951cf None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:53.593682 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada7d15-c06d-4dd5-a457-c0a18f2951cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:53.593985 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada7d15-c06d-4dd5-a457-c0a18f2951cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:53.595379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada7d15-c06d-4dd5-a457-c0a18f2951cf None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:53.603643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 536/1088] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:06:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:53.609520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2057fb5-2b54-4e84-a051-77b0b5782176 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:53.625274 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:53.719149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2057fb5-2b54-4e84-a051-77b0b5782176 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:53.753286 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b852428-6e9c-4137-9221-aed812c5a14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:53.753921 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b852428-6e9c-4137-9221-aed812c5a14e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:53.754203 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b852428-6e9c-4137-9221-aed812c5a14e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:53.754479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b852428-6e9c-4137-9221-aed812c5a14e None admin] PATH_INFO: `/v3/users/079271a5f81c4299a730eee474368ada` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:53.756178 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b852428-6e9c-4137-9221-aed812c5a14e None admin] RBAC: Authorizing `identity:get_user(user_id=079271a5f81c4299a730eee474368ada)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:53.757614 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b852428-6e9c-4137-9221-aed812c5a14e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:53.759231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 553/1089] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:06:53 2022] GET /identity/v3/users/079271a5f81c4299a730eee474368ada => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:53.764884 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada7d15-c06d-4dd5-a457-c0a18f2951cf None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:53.862435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada7d15-c06d-4dd5-a457-c0a18f2951cf None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:53.875495 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:53.896285 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1b35b-9b39-4735-bb81-d3731f88e283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:53.897029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1b35b-9b39-4735-bb81-d3731f88e283 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:53.897314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1b35b-9b39-4735-bb81-d3731f88e283 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:53.897513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1b35b-9b39-4735-bb81-d3731f88e283 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:53.984303 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1b35b-9b39-4735-bb81-d3731f88e283 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:53.985772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1b35b-9b39-4735-bb81-d3731f88e283 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:54.085466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 537/1090] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:06:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 11:06:54.091149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1b35b-9b39-4735-bb81-d3731f88e283 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:54.092046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb0585-5a06-40fd-81ee-3cdd471a418c None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:54.092489 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb0585-5a06-40fd-81ee-3cdd471a418c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:54.092716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb0585-5a06-40fd-81ee-3cdd471a418c None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:54.094043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 554/1091] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:54.099858 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363542e3-e93e-48e3-b8d0-5eeeb21a4916 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:54.100137 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363542e3-e93e-48e3-b8d0-5eeeb21a4916 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:54.100407 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363542e3-e93e-48e3-b8d0-5eeeb21a4916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:54.105684 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:54.121649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-363542e3-e93e-48e3-b8d0-5eeeb21a4916 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:54.122853 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-363542e3-e93e-48e3-b8d0-5eeeb21a4916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:54.190139 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 555/1092] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:06:54.195471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1b35b-9b39-4735-bb81-d3731f88e283 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:54.229874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b250b9-5d84-4f8f-acd2-09285e89db4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:54.230503 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b250b9-5d84-4f8f-acd2-09285e89db4a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:54.230783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b250b9-5d84-4f8f-acd2-09285e89db4a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:54.231060 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b250b9-5d84-4f8f-acd2-09285e89db4a None admin] PATH_INFO: `/v3/projects/c632b8ec86394fa9b71d93af3293f05d` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:54.232194 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b250b9-5d84-4f8f-acd2-09285e89db4a None admin] RBAC: Authorizing `identity:get_project(project_id=c632b8ec86394fa9b71d93af3293f05d)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:54.233621 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b250b9-5d84-4f8f-acd2-09285e89db4a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:54.240783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 538/1093] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:06:54 2022] GET /identity/v3/projects/c632b8ec86394fa9b71d93af3293f05d => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:06:54.243195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363542e3-e93e-48e3-b8d0-5eeeb21a4916 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:54.262311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:54.348467 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363542e3-e93e-48e3-b8d0-5eeeb21a4916 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:54.382145 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a309b12-1926-46b6-a01b-4714f5aced27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:54.382673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a309b12-1926-46b6-a01b-4714f5aced27 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:54.382954 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a309b12-1926-46b6-a01b-4714f5aced27 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:54.383254 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a309b12-1926-46b6-a01b-4714f5aced27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:54.383975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a309b12-1926-46b6-a01b-4714f5aced27 None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=c632b8ec86394fa9b71d93af3293f05d, user.id=079271a5f81c4299a730eee474368ada) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:54.384282 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a309b12-1926-46b6-a01b-4714f5aced27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:54.385697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a309b12-1926-46b6-a01b-4714f5aced27 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:54.391344 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 556/1094] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:06:54 2022] GET /identity/v3/role_assignments?user.id=079271a5f81c4299a730eee474368ada&scope.project.id=c632b8ec86394fa9b71d93af3293f05d&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:54.473943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b250b9-5d84-4f8f-acd2-09285e89db4a None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:54.560691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:54.580225 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b250b9-5d84-4f8f-acd2-09285e89db4a None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:54.615672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0733a-aa78-4414-8e3b-21317fa104c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:54.616320 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0733a-aa78-4414-8e3b-21317fa104c1 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:54.616605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0733a-aa78-4414-8e3b-21317fa104c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:54.616874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0733a-aa78-4414-8e3b-21317fa104c1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:54.627663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0733a-aa78-4414-8e3b-21317fa104c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:06:54.632636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-5bb0733a-aa78-4414-8e3b-21317fa104c1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:06:54.632636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:54.632636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:54.632636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:54.632636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:54.632636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:54.632636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:54.632636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:54.632636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:54.632636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:54.632636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:54.632636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:54.632636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:06:54.632636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:06:54.632636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:06:54.632636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:06:54.633570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:06:54.633570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:06:54.633570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:06:54.633570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:06:54.633570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:54.633570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:54.633570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:54.633570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:54.633570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:54.633570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:54.633570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:54.633570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:54.633570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:54.633570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:54.633570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:54.633570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:54.634414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:54.634414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:54.634414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:54.634414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:54.634414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:06:54.634414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:06:54.634414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:06:54.634414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:06:54.634414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:06:54.634414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:06:54.634414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:06:54.634414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:54.634414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 539/1095] 10.222.0.78 () {56 vars in 1120 bytes} [Mon Sep 19 11:06:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:54.639646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a309b12-1926-46b6-a01b-4714f5aced27 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:54.715019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:54.739042 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a309b12-1926-46b6-a01b-4714f5aced27 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:54.774286 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96860b7-0d48-4b57-ae2c-6b462e22adca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:54.774899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96860b7-0d48-4b57-ae2c-6b462e22adca None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:54.775193 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96860b7-0d48-4b57-ae2c-6b462e22adca None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:54.775492 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96860b7-0d48-4b57-ae2c-6b462e22adca None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:54.776161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96860b7-0d48-4b57-ae2c-6b462e22adca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:54.776466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96860b7-0d48-4b57-ae2c-6b462e22adca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:54.777835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96860b7-0d48-4b57-ae2c-6b462e22adca None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:54.786148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 557/1096] 10.222.0.78 () {56 vars in 1115 bytes} [Mon Sep 19 11:06:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:54.792564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0733a-aa78-4414-8e3b-21317fa104c1 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:54.891829 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0733a-aa78-4414-8e3b-21317fa104c1 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:54.925721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214bfe5-4f17-4d00-92e4-f0dfcb04a100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:54.926344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214bfe5-4f17-4d00-92e4-f0dfcb04a100 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:54.926622 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214bfe5-4f17-4d00-92e4-f0dfcb04a100 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:54.926896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214bfe5-4f17-4d00-92e4-f0dfcb04a100 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:54.935729 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:54.938191 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214bfe5-4f17-4d00-92e4-f0dfcb04a100 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:54.939684 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214bfe5-4f17-4d00-92e4-f0dfcb04a100 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:54.954472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-8214bfe5-4f17-4d00-92e4-f0dfcb04a100 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:54.954472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:54.954472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:54.954472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:54.954472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:54.954472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:54.954472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:54.954472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:54.954472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:54.954472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:54.954472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:54.954472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:54.954472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:54.954472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:54.954472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:54.954472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:54.955494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:54.955494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:54.955494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:54.955494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:54.955494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:54.955494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:54.955494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:54.955494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:54.955494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:54.955494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:54.955494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:54.955494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:54.955494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:54.955494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:54.955494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:54.955494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:54.955494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:54.956411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:54.956411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:54.956411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:54.956411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 540/1097] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:54.962029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96860b7-0d48-4b57-ae2c-6b462e22adca None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:55.058080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96860b7-0d48-4b57-ae2c-6b462e22adca None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:55.091137 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2faff-d061-47b8-84ca-969cb700486c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:55.091774 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2faff-d061-47b8-84ca-969cb700486c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:55.092055 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2faff-d061-47b8-84ca-969cb700486c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:55.092615 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2faff-d061-47b8-84ca-969cb700486c None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:55.092945 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2faff-d061-47b8-84ca-969cb700486c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:55.093246 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2faff-d061-47b8-84ca-969cb700486c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:55.094598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2faff-d061-47b8-84ca-969cb700486c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:55.096951 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:55.107353 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 558/1098] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:55.112085 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214bfe5-4f17-4d00-92e4-f0dfcb04a100 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:55.212783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214bfe5-4f17-4d00-92e4-f0dfcb04a100 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:55.245562 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd2e0e-b561-42fd-9a43-bc96370c7eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:55.246186 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd2e0e-b561-42fd-9a43-bc96370c7eee None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:55.246460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd2e0e-b561-42fd-9a43-bc96370c7eee None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:55.246728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd2e0e-b561-42fd-9a43-bc96370c7eee None admin] PATH_INFO: `/v3/users/nova` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:55.254241 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd2e0e-b561-42fd-9a43-bc96370c7eee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:55.256300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd2e0e-b561-42fd-9a43-bc96370c7eee None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:55.267828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-d5bd2e0e-b561-42fd-9a43-bc96370c7eee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:06:55.267828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:55.267828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:55.267828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:55.267828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:55.267828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:55.267828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:55.267828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:55.267828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:55.267828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:55.267828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:55.267828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:55.267828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:06:55.267828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:06:55.267828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:06:55.267828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:55.269010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:55.269971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:55.269971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:55.269971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:06:55.269971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:06:55.269971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:06:55.269971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:06:55.269971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:06:55.269971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:06:55.269971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:06:55.269971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:06:55.269971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 541/1099] 10.222.0.78 () {56 vars in 1093 bytes} [Mon Sep 19 11:06:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:55.274702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2faff-d061-47b8-84ca-969cb700486c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:55.378493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2faff-d061-47b8-84ca-969cb700486c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:55.412429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8db22-d979-460d-93c9-55e16cc8d49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:55.413028 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8db22-d979-460d-93c9-55e16cc8d49c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:55.413305 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8db22-d979-460d-93c9-55e16cc8d49c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:55.413576 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8db22-d979-460d-93c9-55e16cc8d49c None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:55.414296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8db22-d979-460d-93c9-55e16cc8d49c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:55.414598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8db22-d979-460d-93c9-55e16cc8d49c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:55.416280 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8db22-d979-460d-93c9-55e16cc8d49c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:55.434333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 559/1100] 10.222.0.78 () {56 vars in 1133 bytes} [Mon Sep 19 11:06:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:55.440120 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd2e0e-b561-42fd-9a43-bc96370c7eee None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:55.493367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:55.540035 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd2e0e-b561-42fd-9a43-bc96370c7eee None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:55.573594 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c44bc5-8bf5-48fd-a988-3ec362ede300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:55.574137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c44bc5-8bf5-48fd-a988-3ec362ede300 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:55.574436 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c44bc5-8bf5-48fd-a988-3ec362ede300 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:55.574707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c44bc5-8bf5-48fd-a988-3ec362ede300 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:55.641022 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c44bc5-8bf5-48fd-a988-3ec362ede300 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:55.642461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c44bc5-8bf5-48fd-a988-3ec362ede300 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:55.692303 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:55.738819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 542/1101] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:06:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:06:55.745057 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8db22-d979-460d-93c9-55e16cc8d49c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:55.799059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:55.841760 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8db22-d979-460d-93c9-55e16cc8d49c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:55.875876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d674a2-3030-4251-aa3c-095e534c5458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:55.876508 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d674a2-3030-4251-aa3c-095e534c5458 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:55.876787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d674a2-3030-4251-aa3c-095e534c5458 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:55.877058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d674a2-3030-4251-aa3c-095e534c5458 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:55.887616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d674a2-3030-4251-aa3c-095e534c5458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:55.889100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d674a2-3030-4251-aa3c-095e534c5458 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:55.902791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-52d674a2-3030-4251-aa3c-095e534c5458 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:55.902791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:55.902791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:55.902791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:55.902791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:55.902791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:55.902791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:55.902791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:55.902791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:55.902791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:55.902791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:55.902791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:55.902791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:06:55.902791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:06:55.902791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:06:55.902791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:06:55.903934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:55.903934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:55.903934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:55.903934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:55.903934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:55.903934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:55.903934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:55.903934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:55.903934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:55.903934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:55.903934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:55.903934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:55.903934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:55.903934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:55.903934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:55.903934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:55.903934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:06:55.904841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:06:55.904841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:06:55.904841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:55.904841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 560/1102] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:55.909574 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c44bc5-8bf5-48fd-a988-3ec362ede300 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:56.007837 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c44bc5-8bf5-48fd-a988-3ec362ede300 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:56.041689 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2d7b5-cedc-4391-89c3-1c0be1d1d23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:56.042299 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2d7b5-cedc-4391-89c3-1c0be1d1d23d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:56.042579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2d7b5-cedc-4391-89c3-1c0be1d1d23d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:56.042857 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2d7b5-cedc-4391-89c3-1c0be1d1d23d None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:56.043537 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2d7b5-cedc-4391-89c3-1c0be1d1d23d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:56.043842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2d7b5-cedc-4391-89c3-1c0be1d1d23d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:56.045201 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2d7b5-cedc-4391-89c3-1c0be1d1d23d None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:56.056362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 543/1103] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:56.061924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d674a2-3030-4251-aa3c-095e534c5458 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:56.164675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d674a2-3030-4251-aa3c-095e534c5458 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:56.171940 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:56.197909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712037a-1216-4787-a778-37bd75c7cf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:56.198541 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712037a-1216-4787-a778-37bd75c7cf7b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:56.198822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712037a-1216-4787-a778-37bd75c7cf7b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:56.199093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712037a-1216-4787-a778-37bd75c7cf7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:56.205746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712037a-1216-4787-a778-37bd75c7cf7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:56.207595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712037a-1216-4787-a778-37bd75c7cf7b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:56.219693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-2712037a-1216-4787-a778-37bd75c7cf7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:56.219693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:06:56.219693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:06:56.219693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:06:56.219693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:06:56.219693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:06:56.219693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:06:56.219693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:06:56.219693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:06:56.219693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:06:56.219693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:06:56.219693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:06:56.219693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:06:56.219693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:06:56.219693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:06:56.220696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:06:56.220696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:06:56.220696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:06:56.220696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:06:56.220696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:06:56.220696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:06:56.220696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:06:56.220696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:06:56.220696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:06:56.220696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:06:56.220696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:06:56.220696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:06:56.220696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:06:56.220696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:06:56.220696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:06:56.220696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:06:56.220696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:06:56.221590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:06:56.221590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:06:56.221590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:06:56.221590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:06:56.221590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:06:56.221590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:06:56.221590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:06:56.221590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:06:56.221590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 561/1104] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:06:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:06:56.227406 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2d7b5-cedc-4391-89c3-1c0be1d1d23d None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:56.247111 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:56.341483 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2d7b5-cedc-4391-89c3-1c0be1d1d23d None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:56.375345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c1804-3922-4063-8070-05c1717fd1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:56.375951 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c1804-3922-4063-8070-05c1717fd1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:56.376233 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c1804-3922-4063-8070-05c1717fd1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:56.376510 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c1804-3922-4063-8070-05c1717fd1f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:56.377196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c1804-3922-4063-8070-05c1717fd1f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:56.377501 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c1804-3922-4063-8070-05c1717fd1f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:56.378928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c1804-3922-4063-8070-05c1717fd1f9 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:56.390707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 544/1105] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:06:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:56.397070 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712037a-1216-4787-a778-37bd75c7cf7b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:56.502486 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712037a-1216-4787-a778-37bd75c7cf7b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:56.543521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a2002-3a69-4279-95b9-94ff1d8835b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:56.544231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a2002-3a69-4279-95b9-94ff1d8835b7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:56.544510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a2002-3a69-4279-95b9-94ff1d8835b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:56.544782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a2002-3a69-4279-95b9-94ff1d8835b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:56.545531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a2002-3a69-4279-95b9-94ff1d8835b7 None admin] RBAC: Adding query filter params (role.id=985ecf1b99cb4533a39fad056b155abb, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=aba1968529574fe5b4c5d114c874f428) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:06:56.545838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a2002-3a69-4279-95b9-94ff1d8835b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:56.547284 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a2002-3a69-4279-95b9-94ff1d8835b7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:56.553962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 562/1106] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:06:56 2022] GET /identity/v3/role_assignments?user.id=aba1968529574fe5b4c5d114c874f428&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=985ecf1b99cb4533a39fad056b155abb => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:56.606434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19268ae-b7f5-4cf8-9770-8456a88b9aae None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:56.606647 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19268ae-b7f5-4cf8-9770-8456a88b9aae None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:56.606899 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19268ae-b7f5-4cf8-9770-8456a88b9aae None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:56.608345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 545/1107] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:56.614423 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130031d-68fe-48c8-81b1-74f1ae50d6b4 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:56.615299 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130031d-68fe-48c8-81b1-74f1ae50d6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:56.615583 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130031d-68fe-48c8-81b1-74f1ae50d6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:56.635860 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4130031d-68fe-48c8-81b1-74f1ae50d6b4 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:56.637101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4130031d-68fe-48c8-81b1-74f1ae50d6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:56.663529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:56.711713 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 563/1108] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:56.989931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19268ae-b7f5-4cf8-9770-8456a88b9aae None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:57.069416 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:57.101211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19268ae-b7f5-4cf8-9770-8456a88b9aae None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:57.141436 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789904f-678f-48a4-8163-dfaa63eda5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:57.142074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789904f-678f-48a4-8163-dfaa63eda5b5 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:57.142355 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789904f-678f-48a4-8163-dfaa63eda5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:57.142629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789904f-678f-48a4-8163-dfaa63eda5b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:57.143815 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e789904f-678f-48a4-8163-dfaa63eda5b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:57.145264 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e789904f-678f-48a4-8163-dfaa63eda5b5 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:57.147510 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 546/1109] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:06:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:06:57.153276 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130031d-68fe-48c8-81b1-74f1ae50d6b4 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:06:57.259692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130031d-68fe-48c8-81b1-74f1ae50d6b4 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:06:57.294573 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490c3f7-91d2-4ecf-9f2b-de455fa3817c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:06:57.295219 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490c3f7-91d2-4ecf-9f2b-de455fa3817c None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:57.295511 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490c3f7-91d2-4ecf-9f2b-de455fa3817c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:57.295787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490c3f7-91d2-4ecf-9f2b-de455fa3817c None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:57.296452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490c3f7-91d2-4ecf-9f2b-de455fa3817c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:06:57.297872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490c3f7-91d2-4ecf-9f2b-de455fa3817c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:06:57.384987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 564/1110] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:06:57 2022] POST /identity/v3/users => generated 286 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:06:59.580360 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3d456-ad0b-4c0c-b284-8839bb19c243 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:59.580912 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3d456-ad0b-4c0c-b284-8839bb19c243 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:59.580912 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3d456-ad0b-4c0c-b284-8839bb19c243 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:59.582239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 547/1111] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:06:59.587435 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a03196-e3ac-464e-afd0-7d30ad4a2b13 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:06:59.587872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a03196-e3ac-464e-afd0-7d30ad4a2b13 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:06:59.588053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a03196-e3ac-464e-afd0-7d30ad4a2b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:06:59.606045 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-81a03196-e3ac-464e-afd0-7d30ad4a2b13 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:06:59.607320 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-81a03196-e3ac-464e-afd0-7d30ad4a2b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:06:59.618151 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:06:59.671871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 565/1112] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:06:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:06:59.941654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3d456-ad0b-4c0c-b284-8839bb19c243 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:00.047832 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3d456-ad0b-4c0c-b284-8839bb19c243 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:00.081255 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db20ff-d660-41c2-ac14-0d8db7d9554e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:00.081880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db20ff-d660-41c2-ac14-0d8db7d9554e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:00.082169 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db20ff-d660-41c2-ac14-0d8db7d9554e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:00.082445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db20ff-d660-41c2-ac14-0d8db7d9554e None admin] PATH_INFO: `/v3/roles/b7eccd60e28d4b39b54879dad8230e0d` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:00.083872 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db20ff-d660-41c2-ac14-0d8db7d9554e None admin] RBAC: Authorizing `identity:get_role(role_id=b7eccd60e28d4b39b54879dad8230e0d)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:00.085239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db20ff-d660-41c2-ac14-0d8db7d9554e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:00.086560 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 548/1113] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:06:59 2022] GET /identity/v3/roles/b7eccd60e28d4b39b54879dad8230e0d => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:00.092462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a03196-e3ac-464e-afd0-7d30ad4a2b13 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:00.190340 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a03196-e3ac-464e-afd0-7d30ad4a2b13 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:00.224590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c5788-f449-432b-9bee-61d54052f7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:00.225133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c5788-f449-432b-9bee-61d54052f7da None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:00.225414 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c5788-f449-432b-9bee-61d54052f7da None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:00.225685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c5788-f449-432b-9bee-61d54052f7da None admin] PATH_INFO: `/v3/users/d862111f4ef54dfa9ccc878d03871363` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:00.238179 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c5788-f449-432b-9bee-61d54052f7da None admin] RBAC: Authorizing `identity:get_user(user_id=d862111f4ef54dfa9ccc878d03871363)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:00.240218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c5788-f449-432b-9bee-61d54052f7da None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:00.241803 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 566/1114] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:07:00 2022] GET /identity/v3/users/d862111f4ef54dfa9ccc878d03871363 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:00.248159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db20ff-d660-41c2-ac14-0d8db7d9554e None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:00.259425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:00.357452 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db20ff-d660-41c2-ac14-0d8db7d9554e None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:00.391534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a14e5-966b-42b8-bc4e-1ff2904a43c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:00.392144 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a14e5-966b-42b8-bc4e-1ff2904a43c0 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:00.392427 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a14e5-966b-42b8-bc4e-1ff2904a43c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:00.392702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a14e5-966b-42b8-bc4e-1ff2904a43c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:00.461766 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a14e5-966b-42b8-bc4e-1ff2904a43c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:00.463228 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a14e5-966b-42b8-bc4e-1ff2904a43c0 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:00.472893 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:00.565342 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 549/1115] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:07:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:07:00.571319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c5788-f449-432b-9bee-61d54052f7da None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:00.575015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:00.757241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c5788-f449-432b-9bee-61d54052f7da None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:00.790995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48724eba-34be-44e5-9373-073096c6c055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:00.791646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48724eba-34be-44e5-9373-073096c6c055 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:00.791924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48724eba-34be-44e5-9373-073096c6c055 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:00.792192 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48724eba-34be-44e5-9373-073096c6c055 None admin] PATH_INFO: `/v3/projects/c632b8ec86394fa9b71d93af3293f05d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:00.793314 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48724eba-34be-44e5-9373-073096c6c055 None admin] RBAC: Authorizing `identity:get_project(project_id=c632b8ec86394fa9b71d93af3293f05d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:00.794719 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48724eba-34be-44e5-9373-073096c6c055 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:00.796361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 567/1116] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:07:00 2022] GET /identity/v3/projects/c632b8ec86394fa9b71d93af3293f05d => generated 311 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:00.802680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a14e5-966b-42b8-bc4e-1ff2904a43c0 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:00.900003 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a14e5-966b-42b8-bc4e-1ff2904a43c0 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:00.908945 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:00.932730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43b685-2744-47cd-8704-bde913ed1d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:00.937324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43b685-2744-47cd-8704-bde913ed1d57 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:00.937324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43b685-2744-47cd-8704-bde913ed1d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:00.937324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43b685-2744-47cd-8704-bde913ed1d57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:00.937324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43b685-2744-47cd-8704-bde913ed1d57 None admin] RBAC: Adding query filter params (role.id=b7eccd60e28d4b39b54879dad8230e0d, scope.project.id=c632b8ec86394fa9b71d93af3293f05d, user.id=d862111f4ef54dfa9ccc878d03871363) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:07:00.937324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43b685-2744-47cd-8704-bde913ed1d57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:00.937324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43b685-2744-47cd-8704-bde913ed1d57 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:00.941750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 550/1117] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:07:00 2022] GET /identity/v3/role_assignments?user.id=d862111f4ef54dfa9ccc878d03871363&scope.project.id=c632b8ec86394fa9b71d93af3293f05d&role.id=b7eccd60e28d4b39b54879dad8230e0d => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:03.055596 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899ae1a-c6bd-4d10-a0d2-9722ed273267 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:03.056158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899ae1a-c6bd-4d10-a0d2-9722ed273267 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:03.056158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899ae1a-c6bd-4d10-a0d2-9722ed273267 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:03.057483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 568/1118] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:07:03.063271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dbae3-f40d-4d22-8a26-38f722992c59 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:03.063745 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dbae3-f40d-4d22-8a26-38f722992c59 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:03.063856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dbae3-f40d-4d22-8a26-38f722992c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:03.082390 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-113dbae3-f40d-4d22-8a26-38f722992c59 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:07:03.083639 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-113dbae3-f40d-4d22-8a26-38f722992c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:07:03.150065 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 551/1119] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:07:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:07:03.424755 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899ae1a-c6bd-4d10-a0d2-9722ed273267 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:03.494640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:03.524402 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899ae1a-c6bd-4d10-a0d2-9722ed273267 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:03.557999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fcb7b-d714-441f-88ff-1e88431224f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:03.558625 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fcb7b-d714-441f-88ff-1e88431224f3 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:03.558908 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fcb7b-d714-441f-88ff-1e88431224f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:03.559204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fcb7b-d714-441f-88ff-1e88431224f3 None admin] PATH_INFO: `/v3/roles/b7eccd60e28d4b39b54879dad8230e0d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:03.560606 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23fcb7b-d714-441f-88ff-1e88431224f3 None admin] RBAC: Authorizing `identity:get_role(role_id=b7eccd60e28d4b39b54879dad8230e0d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:03.561972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23fcb7b-d714-441f-88ff-1e88431224f3 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:03.563383 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 569/1120] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:07:03 2022] GET /identity/v3/roles/b7eccd60e28d4b39b54879dad8230e0d => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:03.569887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113dbae3-f40d-4d22-8a26-38f722992c59 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:03.672096 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113dbae3-f40d-4d22-8a26-38f722992c59 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:03.673604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:03.706708 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92f305-af1d-4f39-bda1-4084a5580455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:03.707293 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92f305-af1d-4f39-bda1-4084a5580455 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:03.707576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92f305-af1d-4f39-bda1-4084a5580455 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:03.707879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92f305-af1d-4f39-bda1-4084a5580455 None admin] PATH_INFO: `/v3/users/d862111f4ef54dfa9ccc878d03871363` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:03.708999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92f305-af1d-4f39-bda1-4084a5580455 None admin] RBAC: Authorizing `identity:get_user(user_id=d862111f4ef54dfa9ccc878d03871363)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:03.710421 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92f305-af1d-4f39-bda1-4084a5580455 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:03.712029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 552/1121] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:07:03 2022] GET /identity/v3/users/d862111f4ef54dfa9ccc878d03871363 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:03.717880 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fcb7b-d714-441f-88ff-1e88431224f3 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:03.814798 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fcb7b-d714-441f-88ff-1e88431224f3 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:03.848050 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d0621-42b7-4001-b3a2-c5f1d4647998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:03.849208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d0621-42b7-4001-b3a2-c5f1d4647998 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:03.849512 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d0621-42b7-4001-b3a2-c5f1d4647998 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:03.849768 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d0621-42b7-4001-b3a2-c5f1d4647998 None admin] PATH_INFO: `/v3/projects/c632b8ec86394fa9b71d93af3293f05d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:03.850875 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d0621-42b7-4001-b3a2-c5f1d4647998 None admin] RBAC: Authorizing `identity:get_project(project_id=c632b8ec86394fa9b71d93af3293f05d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:03.852328 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d0621-42b7-4001-b3a2-c5f1d4647998 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:03.853965 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 570/1122] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:07:03 2022] GET /identity/v3/projects/c632b8ec86394fa9b71d93af3293f05d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:03.860107 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92f305-af1d-4f39-bda1-4084a5580455 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:03.967003 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92f305-af1d-4f39-bda1-4084a5580455 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:04.002145 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9abc0f-dfd4-4065-97e3-b2a0e5c4ae07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:04.002805 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9abc0f-dfd4-4065-97e3-b2a0e5c4ae07 None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:04.003084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9abc0f-dfd4-4065-97e3-b2a0e5c4ae07 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:04.003377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9abc0f-dfd4-4065-97e3-b2a0e5c4ae07 None admin] PATH_INFO: `/v3/projects/c632b8ec86394fa9b71d93af3293f05d/users/d862111f4ef54dfa9ccc878d03871363/roles/b7eccd60e28d4b39b54879dad8230e0d` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:04.005522 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9abc0f-dfd4-4065-97e3-b2a0e5c4ae07 None admin] RBAC: Authorizing `identity:create_grant(project_id=c632b8ec86394fa9b71d93af3293f05d, user_id=d862111f4ef54dfa9ccc878d03871363, role_id=b7eccd60e28d4b39b54879dad8230e0d)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:04.006950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9abc0f-dfd4-4065-97e3-b2a0e5c4ae07 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:04.077895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 553/1123] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:07:03 2022] PUT /identity/v3/projects/c632b8ec86394fa9b71d93af3293f05d/users/d862111f4ef54dfa9ccc878d03871363/roles/b7eccd60e28d4b39b54879dad8230e0d => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:07:06.233565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545384e-2bbf-4f20-8eea-26b6e0e6845c None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:06.234086 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545384e-2bbf-4f20-8eea-26b6e0e6845c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:06.234270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545384e-2bbf-4f20-8eea-26b6e0e6845c None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:06.235513 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 571/1124] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:07:06.240878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d6233-4f54-4889-b01d-4d5a2712eb00 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:06.241298 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d6233-4f54-4889-b01d-4d5a2712eb00 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:06.241409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d6233-4f54-4889-b01d-4d5a2712eb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:06.259877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-610d6233-4f54-4889-b01d-4d5a2712eb00 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:07:06.261054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-610d6233-4f54-4889-b01d-4d5a2712eb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:07:06.284163 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:06.333839 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 554/1125] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:07:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:07:06.602501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545384e-2bbf-4f20-8eea-26b6e0e6845c None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:06.614443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:06.719467 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545384e-2bbf-4f20-8eea-26b6e0e6845c None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:06.752614 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86b058-ed3a-46b1-91a0-ad53800fc902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:06.753235 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86b058-ed3a-46b1-91a0-ad53800fc902 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:06.753515 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86b058-ed3a-46b1-91a0-ad53800fc902 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:06.753786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86b058-ed3a-46b1-91a0-ad53800fc902 None admin] PATH_INFO: `/v3/roles/b7eccd60e28d4b39b54879dad8230e0d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:06.755804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86b058-ed3a-46b1-91a0-ad53800fc902 None admin] RBAC: Authorizing `identity:get_role(role_id=b7eccd60e28d4b39b54879dad8230e0d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:06.757169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86b058-ed3a-46b1-91a0-ad53800fc902 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:06.758482 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 572/1126] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:07:06 2022] GET /identity/v3/roles/b7eccd60e28d4b39b54879dad8230e0d => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:06.764547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d6233-4f54-4889-b01d-4d5a2712eb00 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:06.863015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d6233-4f54-4889-b01d-4d5a2712eb00 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:06.896747 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2065a7-efe2-4786-836d-91bba4d7e78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:06.897380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2065a7-efe2-4786-836d-91bba4d7e78b None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:06.897661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2065a7-efe2-4786-836d-91bba4d7e78b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:06.897935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2065a7-efe2-4786-836d-91bba4d7e78b None admin] PATH_INFO: `/v3/users/d862111f4ef54dfa9ccc878d03871363` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:06.899056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2065a7-efe2-4786-836d-91bba4d7e78b None admin] RBAC: Authorizing `identity:get_user(user_id=d862111f4ef54dfa9ccc878d03871363)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:06.900521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2065a7-efe2-4786-836d-91bba4d7e78b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:06.902122 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 555/1127] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:07:06 2022] GET /identity/v3/users/d862111f4ef54dfa9ccc878d03871363 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:06.907996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86b058-ed3a-46b1-91a0-ad53800fc902 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:06.982931 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:07.005240 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86b058-ed3a-46b1-91a0-ad53800fc902 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:07.044464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf266eb-8cdb-49cf-8624-7bba9c0f5e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:07.045074 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf266eb-8cdb-49cf-8624-7bba9c0f5e66 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:07.045354 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf266eb-8cdb-49cf-8624-7bba9c0f5e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:07.045626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf266eb-8cdb-49cf-8624-7bba9c0f5e66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:07.112843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf266eb-8cdb-49cf-8624-7bba9c0f5e66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:07.114933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf266eb-8cdb-49cf-8624-7bba9c0f5e66 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:07.186639 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:07.209103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 573/1128] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:07:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:07:07.215136 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2065a7-efe2-4786-836d-91bba4d7e78b None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:07.243568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:07.314296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2065a7-efe2-4786-836d-91bba4d7e78b None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:07.361342 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9e7b6-5e26-4061-9260-1292cb6ac68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:07.362140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9e7b6-5e26-4061-9260-1292cb6ac68a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:07.362424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9e7b6-5e26-4061-9260-1292cb6ac68a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:07.362699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9e7b6-5e26-4061-9260-1292cb6ac68a None admin] PATH_INFO: `/v3/projects/c632b8ec86394fa9b71d93af3293f05d` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:07.363918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9e7b6-5e26-4061-9260-1292cb6ac68a None admin] RBAC: Authorizing `identity:get_project(project_id=c632b8ec86394fa9b71d93af3293f05d)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:07.365282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9e7b6-5e26-4061-9260-1292cb6ac68a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:07.366935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 556/1129] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:07:07 2022] GET /identity/v3/projects/c632b8ec86394fa9b71d93af3293f05d => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:07.373415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf266eb-8cdb-49cf-8624-7bba9c0f5e66 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:07.471500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf266eb-8cdb-49cf-8624-7bba9c0f5e66 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:07.504885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766dfd6d-18ec-4358-aae5-4d0b579fc735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:07.505488 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766dfd6d-18ec-4358-aae5-4d0b579fc735 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:07.505772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766dfd6d-18ec-4358-aae5-4d0b579fc735 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:07.506045 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766dfd6d-18ec-4358-aae5-4d0b579fc735 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:07.506765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766dfd6d-18ec-4358-aae5-4d0b579fc735 None admin] RBAC: Adding query filter params (role.id=b7eccd60e28d4b39b54879dad8230e0d, scope.project.id=c632b8ec86394fa9b71d93af3293f05d, user.id=d862111f4ef54dfa9ccc878d03871363) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:07:07.507075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766dfd6d-18ec-4358-aae5-4d0b579fc735 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:07.508564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766dfd6d-18ec-4358-aae5-4d0b579fc735 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:07.514279 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 574/1130] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:07:07 2022] GET /identity/v3/role_assignments?user.id=d862111f4ef54dfa9ccc878d03871363&scope.project.id=c632b8ec86394fa9b71d93af3293f05d&role.id=b7eccd60e28d4b39b54879dad8230e0d => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:09.709790 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03efbc-e1dd-4831-9509-72bfb39f7c5a None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:09.710353 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03efbc-e1dd-4831-9509-72bfb39f7c5a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:09.710353 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03efbc-e1dd-4831-9509-72bfb39f7c5a None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:09.711687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 557/1131] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:07:09.717035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7d1a1-54d1-4803-8f67-e4fa87a190ae None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:09.717413 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7d1a1-54d1-4803-8f67-e4fa87a190ae None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:09.717585 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7d1a1-54d1-4803-8f67-e4fa87a190ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:09.735894 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-34f7d1a1-54d1-4803-8f67-e4fa87a190ae None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:07:09.737104 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-34f7d1a1-54d1-4803-8f67-e4fa87a190ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:07:09.770564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:09.802405 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 575/1132] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:07:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:07:10.066710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03efbc-e1dd-4831-9509-72bfb39f7c5a None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:10.164604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:10.171549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03efbc-e1dd-4831-9509-72bfb39f7c5a None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:10.206243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc684f-9542-4b54-9c73-3c67f3696c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:10.206886 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc684f-9542-4b54-9c73-3c67f3696c60 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:10.207241 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc684f-9542-4b54-9c73-3c67f3696c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:10.207498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc684f-9542-4b54-9c73-3c67f3696c60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:10.209116 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc684f-9542-4b54-9c73-3c67f3696c60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:10.210638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc684f-9542-4b54-9c73-3c67f3696c60 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:10.212417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 558/1133] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:07:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:10.218362 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7d1a1-54d1-4803-8f67-e4fa87a190ae None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:10.320482 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7d1a1-54d1-4803-8f67-e4fa87a190ae None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:10.357141 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a973d7-852f-47de-a8e9-c2e660c45229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:10.357744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a973d7-852f-47de-a8e9-c2e660c45229 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:10.358022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a973d7-852f-47de-a8e9-c2e660c45229 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:10.358292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a973d7-852f-47de-a8e9-c2e660c45229 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:10.358944 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a973d7-852f-47de-a8e9-c2e660c45229 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:10.360392 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a973d7-852f-47de-a8e9-c2e660c45229 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:10.438861 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 576/1134] 10.222.0.78 () {60 vars in 1141 bytes} [Mon Sep 19 11:07:10 2022] POST /identity/v3/projects => generated 311 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:07:12.631046 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26de97d-a62a-4b7d-bb25-eb2884cb3112 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:12.631689 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26de97d-a62a-4b7d-bb25-eb2884cb3112 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:12.631689 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26de97d-a62a-4b7d-bb25-eb2884cb3112 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:12.633039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 559/1135] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:07:12.638328 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff226e83-7ba4-42d0-a137-e50414e760ab None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:12.638746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff226e83-7ba4-42d0-a137-e50414e760ab None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:12.638883 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff226e83-7ba4-42d0-a137-e50414e760ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:12.656889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ff226e83-7ba4-42d0-a137-e50414e760ab None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:07:12.658081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ff226e83-7ba4-42d0-a137-e50414e760ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:07:12.674816 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:12.726884 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 577/1136] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:07:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:07:12.997340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26de97d-a62a-4b7d-bb25-eb2884cb3112 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:13.097609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26de97d-a62a-4b7d-bb25-eb2884cb3112 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:13.132896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbd346-b865-4516-8d76-a75ba1eb22f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:13.133526 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd346-b865-4516-8d76-a75ba1eb22f4 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:13.133806 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd346-b865-4516-8d76-a75ba1eb22f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:13.134081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd346-b865-4516-8d76-a75ba1eb22f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:13.135213 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbd346-b865-4516-8d76-a75ba1eb22f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:13.136757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbd346-b865-4516-8d76-a75ba1eb22f4 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:13.138340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 560/1137] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:07:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:13.144114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff226e83-7ba4-42d0-a137-e50414e760ab None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:13.242649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff226e83-7ba4-42d0-a137-e50414e760ab None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:13.276336 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89440f-4dbe-4db9-84e3-c786d85dbe44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:13.276943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89440f-4dbe-4db9-84e3-c786d85dbe44 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:13.277226 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89440f-4dbe-4db9-84e3-c786d85dbe44 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:13.277439 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89440f-4dbe-4db9-84e3-c786d85dbe44 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:13.278085 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c89440f-4dbe-4db9-84e3-c786d85dbe44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:13.279553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c89440f-4dbe-4db9-84e3-c786d85dbe44 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:13.397998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 578/1138] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:07:13 2022] POST /identity/v3/users => generated 286 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:07:15.612847 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca6ba2-2671-4f4d-986b-b4ac1c5317ba None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:15.613382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca6ba2-2671-4f4d-986b-b4ac1c5317ba None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:15.613475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca6ba2-2671-4f4d-986b-b4ac1c5317ba None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:15.615402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 561/1139] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:07:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:07:15.620769 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123ecd9-a963-4f40-836a-dc7f1ba19f91 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:15.621169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123ecd9-a963-4f40-836a-dc7f1ba19f91 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:15.621334 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123ecd9-a963-4f40-836a-dc7f1ba19f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:15.639443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4123ecd9-a963-4f40-836a-dc7f1ba19f91 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:07:15.640662 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4123ecd9-a963-4f40-836a-dc7f1ba19f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:07:15.642601 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:15.711735 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 579/1140] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:07:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:07:15.977687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca6ba2-2671-4f4d-986b-b4ac1c5317ba None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:16.008499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:16.080345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca6ba2-2671-4f4d-986b-b4ac1c5317ba None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:16.114556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968adcf3-a644-4bd8-b1a0-6e01f964a85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:16.115216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968adcf3-a644-4bd8-b1a0-6e01f964a85f None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:16.115499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968adcf3-a644-4bd8-b1a0-6e01f964a85f None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:16.115768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968adcf3-a644-4bd8-b1a0-6e01f964a85f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:16.126122 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968adcf3-a644-4bd8-b1a0-6e01f964a85f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:07:16.130928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-968adcf3-a644-4bd8-b1a0-6e01f964a85f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:07:16.130928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:07:16.130928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:07:16.130928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:07:16.130928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:07:16.130928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:07:16.130928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:07:16.130928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:07:16.130928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:07:16.130928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:07:16.130928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:07:16.130928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:07:16.130928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:07:16.130928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:07:16.130928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:07:16.130928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:07:16.132049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:07:16.132049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:07:16.132049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:07:16.132049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:07:16.132049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:07:16.132049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:07:16.132049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:07:16.132049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:07:16.132049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:07:16.132049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:07:16.132049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:07:16.132049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:07:16.132049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:07:16.132049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:07:16.132049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:07:16.132049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:07:16.132947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:07:16.132947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:07:16.132947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:07:16.132947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:07:16.132947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:07:16.132947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:07:16.132947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:07:16.132947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:07:16.132947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:07:16.132947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:07:16.132947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:07:16.132947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:07:16.132947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 562/1141] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:07:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:07:16.138086 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123ecd9-a963-4f40-836a-dc7f1ba19f91 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:16.234800 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123ecd9-a963-4f40-836a-dc7f1ba19f91 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:16.267973 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa130ba-b2c4-483e-8ca2-96feb38ad969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:16.268576 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa130ba-b2c4-483e-8ca2-96feb38ad969 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:16.268855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa130ba-b2c4-483e-8ca2-96feb38ad969 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:16.269127 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa130ba-b2c4-483e-8ca2-96feb38ad969 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:16.270419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa130ba-b2c4-483e-8ca2-96feb38ad969 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:07:16.270727 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa130ba-b2c4-483e-8ca2-96feb38ad969 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:16.272093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa130ba-b2c4-483e-8ca2-96feb38ad969 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:16.276943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:16.280216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 580/1142] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:07:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:16.286229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968adcf3-a644-4bd8-b1a0-6e01f964a85f None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:16.378025 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:16.389588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968adcf3-a644-4bd8-b1a0-6e01f964a85f None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:16.425679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f13915-e286-4a3c-9687-7a9fb65554a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:16.426313 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f13915-e286-4a3c-9687-7a9fb65554a6 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:16.426592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f13915-e286-4a3c-9687-7a9fb65554a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:16.426866 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f13915-e286-4a3c-9687-7a9fb65554a6 None admin] PATH_INFO: `/v3/users/3e0790d4e7fb4c76999cef3bd999dba6` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:16.440270 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f13915-e286-4a3c-9687-7a9fb65554a6 None admin] RBAC: Authorizing `identity:get_user(user_id=3e0790d4e7fb4c76999cef3bd999dba6)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:16.442221 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f13915-e286-4a3c-9687-7a9fb65554a6 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:16.443873 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 563/1143] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:07:16 2022] GET /identity/v3/users/3e0790d4e7fb4c76999cef3bd999dba6 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:16.449828 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa130ba-b2c4-483e-8ca2-96feb38ad969 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:16.548188 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa130ba-b2c4-483e-8ca2-96feb38ad969 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:16.581161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360de8e2-0c18-4082-9ac4-3f7281596bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:16.581773 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360de8e2-0c18-4082-9ac4-3f7281596bbe None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:16.582056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360de8e2-0c18-4082-9ac4-3f7281596bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:16.582951 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360de8e2-0c18-4082-9ac4-3f7281596bbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:16.646449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:16.648561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360de8e2-0c18-4082-9ac4-3f7281596bbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:16.650058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360de8e2-0c18-4082-9ac4-3f7281596bbe None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:16.747085 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 581/1144] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:07:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:07:16.753404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f13915-e286-4a3c-9687-7a9fb65554a6 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:16.851566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f13915-e286-4a3c-9687-7a9fb65554a6 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:16.886040 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb0107-5a94-45d0-92ad-3d4e4edf7a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:16.886691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb0107-5a94-45d0-92ad-3d4e4edf7a7a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:16.886972 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb0107-5a94-45d0-92ad-3d4e4edf7a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:16.887272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb0107-5a94-45d0-92ad-3d4e4edf7a7a None admin] PATH_INFO: `/v3/projects/92656dc1e0b34bec8ea7498e58da6004` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:16.888614 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb0107-5a94-45d0-92ad-3d4e4edf7a7a None admin] RBAC: Authorizing `identity:get_project(project_id=92656dc1e0b34bec8ea7498e58da6004)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:16.889814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb0107-5a94-45d0-92ad-3d4e4edf7a7a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:16.891499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 564/1145] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:07:16 2022] GET /identity/v3/projects/92656dc1e0b34bec8ea7498e58da6004 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:16.897819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360de8e2-0c18-4082-9ac4-3f7281596bbe None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:16.990361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:16.993272 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360de8e2-0c18-4082-9ac4-3f7281596bbe None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:17.027016 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c9c21-ffa2-4227-a070-41eaa40f9159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:17.027646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c9c21-ffa2-4227-a070-41eaa40f9159 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:17.027928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c9c21-ffa2-4227-a070-41eaa40f9159 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:17.028202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c9c21-ffa2-4227-a070-41eaa40f9159 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:17.028922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c9c21-ffa2-4227-a070-41eaa40f9159 None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=92656dc1e0b34bec8ea7498e58da6004, user.id=3e0790d4e7fb4c76999cef3bd999dba6) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:07:17.029232 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c9c21-ffa2-4227-a070-41eaa40f9159 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:17.030743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c9c21-ffa2-4227-a070-41eaa40f9159 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:17.037382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 582/1146] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:07:16 2022] GET /identity/v3/role_assignments?user.id=3e0790d4e7fb4c76999cef3bd999dba6&scope.project.id=92656dc1e0b34bec8ea7498e58da6004&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:19.131779 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f7173-9f26-4fee-928d-72f8e0ae75df None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:19.132283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f7173-9f26-4fee-928d-72f8e0ae75df None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:19.132283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f7173-9f26-4fee-928d-72f8e0ae75df None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:19.133635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 565/1147] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:07:19.138716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83745a59-7fdf-406f-95ec-ff451f0c02cc None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:19.139141 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83745a59-7fdf-406f-95ec-ff451f0c02cc None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:19.139298 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83745a59-7fdf-406f-95ec-ff451f0c02cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:19.157983 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-83745a59-7fdf-406f-95ec-ff451f0c02cc None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:07:19.159345 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-83745a59-7fdf-406f-95ec-ff451f0c02cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:07:19.227595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 583/1148] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:07:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:07:19.494360 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f7173-9f26-4fee-928d-72f8e0ae75df None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:19.513753 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:19.594926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f7173-9f26-4fee-928d-72f8e0ae75df None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:19.628646 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da5015-6244-4712-be6b-32b8bb392f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:19.629834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da5015-6244-4712-be6b-32b8bb392f0d None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:19.630117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da5015-6244-4712-be6b-32b8bb392f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:19.630389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da5015-6244-4712-be6b-32b8bb392f0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:19.640257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51da5015-6244-4712-be6b-32b8bb392f0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:07:19.645134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-51da5015-6244-4712-be6b-32b8bb392f0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:07:19.645134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:07:19.645134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:07:19.645134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:07:19.645134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:07:19.645134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:07:19.645134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:07:19.645134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:07:19.645134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:07:19.645134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:07:19.645134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:07:19.645134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:07:19.645134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:07:19.645134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:07:19.645134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:07:19.645134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:07:19.646250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:07:19.646250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:07:19.646250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:07:19.646250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:07:19.646250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:07:19.646250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:07:19.646250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:07:19.646250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:07:19.646250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:07:19.646250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:07:19.646250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:07:19.646250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:07:19.646250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:07:19.646250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:07:19.646250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:07:19.646250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:07:19.647204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:07:19.647204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:07:19.647204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:07:19.647204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:07:19.647204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:07:19.647204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:07:19.647204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:07:19.647204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:07:19.647204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:07:19.647204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:07:19.647204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:07:19.647204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:07:19.647204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 566/1149] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:07:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:07:19.652266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83745a59-7fdf-406f-95ec-ff451f0c02cc None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:19.742058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:19.748798 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83745a59-7fdf-406f-95ec-ff451f0c02cc None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:19.780918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280d5ee-cd10-48ce-8128-d0f2191c99ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:19.781503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280d5ee-cd10-48ce-8128-d0f2191c99ae None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:19.781781 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280d5ee-cd10-48ce-8128-d0f2191c99ae None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:19.782049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280d5ee-cd10-48ce-8128-d0f2191c99ae None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:19.782698 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280d5ee-cd10-48ce-8128-d0f2191c99ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:07:19.782996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280d5ee-cd10-48ce-8128-d0f2191c99ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:19.784381 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280d5ee-cd10-48ce-8128-d0f2191c99ae None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:19.793338 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 584/1150] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:07:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:19.799234 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da5015-6244-4712-be6b-32b8bb392f0d None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:19.876311 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:19.897499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da5015-6244-4712-be6b-32b8bb392f0d None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:19.931589 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97677422-2b8c-4ec7-9be6-702a5e58ef7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:19.932218 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97677422-2b8c-4ec7-9be6-702a5e58ef7b None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:19.932500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97677422-2b8c-4ec7-9be6-702a5e58ef7b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:19.932775 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97677422-2b8c-4ec7-9be6-702a5e58ef7b None admin] PATH_INFO: `/v3/users/3e0790d4e7fb4c76999cef3bd999dba6` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:19.933889 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97677422-2b8c-4ec7-9be6-702a5e58ef7b None admin] RBAC: Authorizing `identity:get_user(user_id=3e0790d4e7fb4c76999cef3bd999dba6)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:19.935356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97677422-2b8c-4ec7-9be6-702a5e58ef7b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:19.936936 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 567/1151] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:07:19 2022] GET /identity/v3/users/3e0790d4e7fb4c76999cef3bd999dba6 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:19.942726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280d5ee-cd10-48ce-8128-d0f2191c99ae None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:20.039915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280d5ee-cd10-48ce-8128-d0f2191c99ae None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:20.071876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcf531-c697-4940-8465-805948849fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:20.072499 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcf531-c697-4940-8465-805948849fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:20.072799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcf531-c697-4940-8465-805948849fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:20.073092 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcf531-c697-4940-8465-805948849fa2 None admin] PATH_INFO: `/v3/projects/92656dc1e0b34bec8ea7498e58da6004` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:20.074157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcf531-c697-4940-8465-805948849fa2 None admin] RBAC: Authorizing `identity:get_project(project_id=92656dc1e0b34bec8ea7498e58da6004)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:20.075633 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcf531-c697-4940-8465-805948849fa2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:20.077207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 585/1152] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:07:19 2022] GET /identity/v3/projects/92656dc1e0b34bec8ea7498e58da6004 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:20.083817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97677422-2b8c-4ec7-9be6-702a5e58ef7b None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:20.180640 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97677422-2b8c-4ec7-9be6-702a5e58ef7b None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:20.214932 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a6439-a398-4b64-a42d-aca729db0f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:20.215620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a6439-a398-4b64-a42d-aca729db0f69 None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:20.215905 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a6439-a398-4b64-a42d-aca729db0f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:20.216181 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a6439-a398-4b64-a42d-aca729db0f69 None admin] PATH_INFO: `/v3/projects/92656dc1e0b34bec8ea7498e58da6004/users/3e0790d4e7fb4c76999cef3bd999dba6/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:20.218089 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a6439-a398-4b64-a42d-aca729db0f69 None admin] RBAC: Authorizing `identity:create_grant(project_id=92656dc1e0b34bec8ea7498e58da6004, user_id=3e0790d4e7fb4c76999cef3bd999dba6, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:20.219589 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a6439-a398-4b64-a42d-aca729db0f69 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:20.311744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 568/1153] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:07:20 2022] PUT /identity/v3/projects/92656dc1e0b34bec8ea7498e58da6004/users/3e0790d4e7fb4c76999cef3bd999dba6/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:07:22.448512 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295f2b0-926c-4355-9cd9-b8ddd2930678 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:22.449071 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295f2b0-926c-4355-9cd9-b8ddd2930678 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:22.449071 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295f2b0-926c-4355-9cd9-b8ddd2930678 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:22.450583 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 586/1154] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:07:22.456544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0af91a-a6d0-4f89-8d08-14ea0dcddce5 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:22.457054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0af91a-a6d0-4f89-8d08-14ea0dcddce5 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:22.457156 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0af91a-a6d0-4f89-8d08-14ea0dcddce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:22.471245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:22.476657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-6b0af91a-a6d0-4f89-8d08-14ea0dcddce5 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:07:22.477847 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-6b0af91a-a6d0-4f89-8d08-14ea0dcddce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:07:22.542100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 569/1155] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:07:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:07:22.814914 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8295f2b0-926c-4355-9cd9-b8ddd2930678 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:22.840257 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:22.912602 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8295f2b0-926c-4355-9cd9-b8ddd2930678 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:22.946243 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b881c-7d48-42be-9152-2ec2cff9600a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:22.946865 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b881c-7d48-42be-9152-2ec2cff9600a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:22.947142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b881c-7d48-42be-9152-2ec2cff9600a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:22.947443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b881c-7d48-42be-9152-2ec2cff9600a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:22.957116 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b881c-7d48-42be-9152-2ec2cff9600a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:07:22.961862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-648b881c-7d48-42be-9152-2ec2cff9600a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:07:22.961862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:07:22.961862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:07:22.961862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:07:22.961862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:07:22.961862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:07:22.961862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:07:22.961862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:07:22.961862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:07:22.961862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:07:22.961862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:07:22.961862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:07:22.961862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:07:22.961862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:07:22.961862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:07:22.961862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:07:22.963094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:07:22.963094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:07:22.963094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:07:22.963094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:07:22.963094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:07:22.963094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:07:22.963094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:07:22.963094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:07:22.963094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:07:22.963094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:07:22.963094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:07:22.963094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:07:22.963094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:07:22.963094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:07:22.963094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:07:22.963094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:07:22.964063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:07:22.964063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:07:22.964063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:07:22.964063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:07:22.964063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:07:22.964063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:07:22.964063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:07:22.964063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:07:22.964063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:07:22.964063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:07:22.964063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:07:22.964063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:07:22.964063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 587/1156] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:07:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:07:22.969054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0af91a-a6d0-4f89-8d08-14ea0dcddce5 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:23.066927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0af91a-a6d0-4f89-8d08-14ea0dcddce5 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:23.100074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892474f-c22c-4d7a-b446-816c20a1b772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:23.100679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892474f-c22c-4d7a-b446-816c20a1b772 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:23.100960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892474f-c22c-4d7a-b446-816c20a1b772 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:23.101230 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892474f-c22c-4d7a-b446-816c20a1b772 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:23.101887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892474f-c22c-4d7a-b446-816c20a1b772 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:07:23.102190 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892474f-c22c-4d7a-b446-816c20a1b772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:23.103581 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892474f-c22c-4d7a-b446-816c20a1b772 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:23.105696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:23.112240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 570/1157] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:07:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:23.118746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b881c-7d48-42be-9152-2ec2cff9600a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:23.201326 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:23.217141 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b881c-7d48-42be-9152-2ec2cff9600a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:23.251746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6576d99-4637-4698-9241-12fce0e1b9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:23.252370 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6576d99-4637-4698-9241-12fce0e1b9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:23.252649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6576d99-4637-4698-9241-12fce0e1b9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:23.252919 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6576d99-4637-4698-9241-12fce0e1b9f4 None admin] PATH_INFO: `/v3/users/3e0790d4e7fb4c76999cef3bd999dba6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:23.254195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6576d99-4637-4698-9241-12fce0e1b9f4 None admin] RBAC: Authorizing `identity:get_user(user_id=3e0790d4e7fb4c76999cef3bd999dba6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:23.255647 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6576d99-4637-4698-9241-12fce0e1b9f4 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:23.257471 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 588/1158] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:07:23 2022] GET /identity/v3/users/3e0790d4e7fb4c76999cef3bd999dba6 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:23.263080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892474f-c22c-4d7a-b446-816c20a1b772 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:23.362633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892474f-c22c-4d7a-b446-816c20a1b772 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:23.398000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add71566-bba5-4603-b74f-37be14aac549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:23.398611 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add71566-bba5-4603-b74f-37be14aac549 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:23.398892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add71566-bba5-4603-b74f-37be14aac549 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:23.399196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add71566-bba5-4603-b74f-37be14aac549 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:23.464007 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:23.467420 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add71566-bba5-4603-b74f-37be14aac549 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:23.468871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add71566-bba5-4603-b74f-37be14aac549 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:23.565113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 571/1159] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:07:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 11:07:23.571158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6576d99-4637-4698-9241-12fce0e1b9f4 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:23.667721 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6576d99-4637-4698-9241-12fce0e1b9f4 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:23.700037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5555544-c59d-41aa-9acd-c5e1cef69e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:23.700664 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5555544-c59d-41aa-9acd-c5e1cef69e92 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:23.700945 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5555544-c59d-41aa-9acd-c5e1cef69e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:23.701219 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5555544-c59d-41aa-9acd-c5e1cef69e92 None admin] PATH_INFO: `/v3/projects/92656dc1e0b34bec8ea7498e58da6004` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:23.702321 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5555544-c59d-41aa-9acd-c5e1cef69e92 None admin] RBAC: Authorizing `identity:get_project(project_id=92656dc1e0b34bec8ea7498e58da6004)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:23.703770 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5555544-c59d-41aa-9acd-c5e1cef69e92 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:23.705388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 589/1160] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:07:23 2022] GET /identity/v3/projects/92656dc1e0b34bec8ea7498e58da6004 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:07:23.712112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add71566-bba5-4603-b74f-37be14aac549 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:23.808354 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:23.812111 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add71566-bba5-4603-b74f-37be14aac549 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:23.847865 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c65f8-b868-4582-8575-e07c1d6b80d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:23.848466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c65f8-b868-4582-8575-e07c1d6b80d2 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:23.848743 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c65f8-b868-4582-8575-e07c1d6b80d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:23.849017 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c65f8-b868-4582-8575-e07c1d6b80d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:23.849734 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c65f8-b868-4582-8575-e07c1d6b80d2 None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=92656dc1e0b34bec8ea7498e58da6004, user.id=3e0790d4e7fb4c76999cef3bd999dba6) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:07:23.850051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c65f8-b868-4582-8575-e07c1d6b80d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:23.851839 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c65f8-b868-4582-8575-e07c1d6b80d2 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:23.857885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 572/1161] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:07:23 2022] GET /identity/v3/role_assignments?user.id=3e0790d4e7fb4c76999cef3bd999dba6&scope.project.id=92656dc1e0b34bec8ea7498e58da6004&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:26.095299 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af81a8c-dfea-488d-b65d-df48228654db None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:26.095863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af81a8c-dfea-488d-b65d-df48228654db None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:26.095863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af81a8c-dfea-488d-b65d-df48228654db None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:26.097184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 590/1162] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:07:26.103231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14d6fb-3d3b-4ab8-ab33-15f109b64614 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:26.103691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14d6fb-3d3b-4ab8-ab33-15f109b64614 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:26.103801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14d6fb-3d3b-4ab8-ab33-15f109b64614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:26.122665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-ef14d6fb-3d3b-4ab8-ab33-15f109b64614 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:07:26.124143 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ef14d6fb-3d3b-4ab8-ab33-15f109b64614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:07:26.188542 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 573/1163] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:07:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:07:26.461582 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af81a8c-dfea-488d-b65d-df48228654db None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:26.489524 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:26.566863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af81a8c-dfea-488d-b65d-df48228654db None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:26.600503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44867002-8a6c-41c1-9aad-f5a438fe41ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:26.601113 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44867002-8a6c-41c1-9aad-f5a438fe41ca None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:26.601399 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44867002-8a6c-41c1-9aad-f5a438fe41ca None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:26.601676 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44867002-8a6c-41c1-9aad-f5a438fe41ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:26.666886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44867002-8a6c-41c1-9aad-f5a438fe41ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:26.668350 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44867002-8a6c-41c1-9aad-f5a438fe41ca None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:26.687889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:26.763443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 591/1164] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:07:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:07:26.770066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14d6fb-3d3b-4ab8-ab33-15f109b64614 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:26.859498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:26.868127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14d6fb-3d3b-4ab8-ab33-15f109b64614 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:26.901914 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00008fa3-5eac-49c8-b40c-0fed2dcb0a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:26.902539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00008fa3-5eac-49c8-b40c-0fed2dcb0a11 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:26.902813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00008fa3-5eac-49c8-b40c-0fed2dcb0a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:26.903079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00008fa3-5eac-49c8-b40c-0fed2dcb0a11 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:26.909641 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00008fa3-5eac-49c8-b40c-0fed2dcb0a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:26.911077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00008fa3-5eac-49c8-b40c-0fed2dcb0a11 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:26.922139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-00008fa3-5eac-49c8-b40c-0fed2dcb0a11 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 19 11:07:26.922139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:07:26.922139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 19 11:07:26.922139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 19 11:07:26.922139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:07:26.922139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:07:26.922139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:07:26.922139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:07:26.922139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 19 11:07:26.922139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:07:26.922139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:07:26.922139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:07:26.922139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:07:26.922139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:07:26.922139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:07:26.922139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:07:26.922139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:07:26.923391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:07:26.923391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:07:26.923391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:07:26.923391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:07:26.923391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:07:26.923391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:07:26.923391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:07:26.923391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:07:26.923391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:07:26.923391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:07:26.923391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:07:26.923391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:07:26.923391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:07:26.923391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:07:26.923391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:07:26.923391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:07:26.924445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:07:26.924445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:07:26.924445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:07:26.924445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:07:26.924445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:07:26.924445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:07:26.924445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:07:26.924445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:07:26.924445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:07:26.924445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:07:26.924445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 19 11:07:26.924445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:07:26.924445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 19 11:07:26.924445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:07:26.924445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 574/1165] 10.222.0.78 () {56 vars in 1117 bytes} [Mon Sep 19 11:07:26 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:07:26.929268 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44867002-8a6c-41c1-9aad-f5a438fe41ca None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:27.024410 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44867002-8a6c-41c1-9aad-f5a438fe41ca None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:27.042815 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:27.055840 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5c1c1-8105-4e69-b2b2-7affd1fff2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:27.056434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5c1c1-8105-4e69-b2b2-7affd1fff2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:27.056711 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5c1c1-8105-4e69-b2b2-7affd1fff2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:27.056980 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5c1c1-8105-4e69-b2b2-7affd1fff2b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:27.057607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5c1c1-8105-4e69-b2b2-7affd1fff2b1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:07:27.057906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5c1c1-8105-4e69-b2b2-7affd1fff2b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:27.059279 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5c1c1-8105-4e69-b2b2-7affd1fff2b1 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:27.067027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 592/1166] 10.222.0.78 () {56 vars in 1115 bytes} [Mon Sep 19 11:07:26 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:29.191279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccc34e-ebae-43c3-8ece-5aae115ddba7 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:29.191828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccc34e-ebae-43c3-8ece-5aae115ddba7 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:29.191828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccc34e-ebae-43c3-8ece-5aae115ddba7 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:29.193159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 575/1167] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:07:29.198343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254fb61-31f9-438b-8a02-1aca97de90cb None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:29.198774 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254fb61-31f9-438b-8a02-1aca97de90cb None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:29.198891 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254fb61-31f9-438b-8a02-1aca97de90cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:29.217302 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c254fb61-31f9-438b-8a02-1aca97de90cb None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:07:29.218795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c254fb61-31f9-438b-8a02-1aca97de90cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:07:29.285322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 593/1168] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:07:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:07:29.553758 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccc34e-ebae-43c3-8ece-5aae115ddba7 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:29.656840 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccc34e-ebae-43c3-8ece-5aae115ddba7 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:29.689983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7371a-0616-4e70-a64e-1a5c7d1a5b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:29.690589 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7371a-0616-4e70-a64e-1a5c7d1a5b54 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:29.690862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7371a-0616-4e70-a64e-1a5c7d1a5b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:29.691747 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7371a-0616-4e70-a64e-1a5c7d1a5b54 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:29.692419 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7371a-0616-4e70-a64e-1a5c7d1a5b54 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:29.693778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7371a-0616-4e70-a64e-1a5c7d1a5b54 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:29.786931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 576/1169] 10.222.0.78 () {60 vars in 1138 bytes} [Mon Sep 19 11:07:29 2022] POST /identity/v3/domains => generated 267 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:07:31.982186 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeb06e-6e97-489a-adf2-654a406c9cfa None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:31.982722 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeb06e-6e97-489a-adf2-654a406c9cfa None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:31.982837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeb06e-6e97-489a-adf2-654a406c9cfa None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:31.984112 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 594/1170] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:07:31.989372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5e9b9-3827-4330-9e32-472dd7ad497e None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:31.989744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5e9b9-3827-4330-9e32-472dd7ad497e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:31.989925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5e9b9-3827-4330-9e32-472dd7ad497e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:32.008890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-08d5e9b9-3827-4330-9e32-472dd7ad497e None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:07:32.010140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-08d5e9b9-3827-4330-9e32-472dd7ad497e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:07:32.017305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:32.076061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 577/1171] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:07:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:07:32.344149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeb06e-6e97-489a-adf2-654a406c9cfa None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:32.445690 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeb06e-6e97-489a-adf2-654a406c9cfa None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:32.460181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:32.479840 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e8fcf-92a3-4c1c-a916-76709ccd4e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:32.481042 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e8fcf-92a3-4c1c-a916-76709ccd4e85 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:32.481042 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e8fcf-92a3-4c1c-a916-76709ccd4e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:32.481042 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e8fcf-92a3-4c1c-a916-76709ccd4e85 None admin] PATH_INFO: `/v3/domains/dcd424585c2d46f4bc3e673c1ae14e71` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:32.492003 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571e8fcf-92a3-4c1c-a916-76709ccd4e85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=dcd424585c2d46f4bc3e673c1ae14e71)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:32.493452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571e8fcf-92a3-4c1c-a916-76709ccd4e85 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:32.503211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 595/1172] 10.222.0.78 () {56 vars in 1183 bytes} [Mon Sep 19 11:07:32 2022] GET /identity/v3/domains/dcd424585c2d46f4bc3e673c1ae14e71 => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:32.510049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5e9b9-3827-4330-9e32-472dd7ad497e None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:32.606282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5e9b9-3827-4330-9e32-472dd7ad497e None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:32.640066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a191b26-7d57-4b75-a905-22bbac78847b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:32.640670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a191b26-7d57-4b75-a905-22bbac78847b None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:32.640952 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a191b26-7d57-4b75-a905-22bbac78847b None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:32.641227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a191b26-7d57-4b75-a905-22bbac78847b None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:32.641868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a191b26-7d57-4b75-a905-22bbac78847b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:32.643327 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a191b26-7d57-4b75-a905-22bbac78847b None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:32.657093 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:32.723515 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 578/1173] 10.222.0.78 () {60 vars in 1141 bytes} [Mon Sep 19 11:07:32 2022] POST /identity/v3/projects => generated 361 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:07:34.896703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b09e4-22cb-4f73-900d-884f547294ab None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:34.897267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b09e4-22cb-4f73-900d-884f547294ab None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:34.897267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b09e4-22cb-4f73-900d-884f547294ab None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:34.898588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 596/1174] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:07:34.904321 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff8245-896c-4f5d-9f4a-16722ec9bddb None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:34.904757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff8245-896c-4f5d-9f4a-16722ec9bddb None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:34.904866 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff8245-896c-4f5d-9f4a-16722ec9bddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:34.923273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-35ff8245-896c-4f5d-9f4a-16722ec9bddb None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:07:34.924463 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-35ff8245-896c-4f5d-9f4a-16722ec9bddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:07:34.988646 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 579/1175] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:07:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:07:35.260204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b09e4-22cb-4f73-900d-884f547294ab None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:35.364609 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b09e4-22cb-4f73-900d-884f547294ab None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:35.398213 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df48619-85cd-455b-a75c-9e937b08191a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:35.398843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df48619-85cd-455b-a75c-9e937b08191a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:35.399127 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df48619-85cd-455b-a75c-9e937b08191a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:35.399434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df48619-85cd-455b-a75c-9e937b08191a None admin] PATH_INFO: `/v3/domains/dcd424585c2d46f4bc3e673c1ae14e71` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:35.400578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df48619-85cd-455b-a75c-9e937b08191a None admin] RBAC: Authorizing `identity:get_domain(domain_id=dcd424585c2d46f4bc3e673c1ae14e71)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:35.402012 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df48619-85cd-455b-a75c-9e937b08191a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:35.403620 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 597/1176] 10.222.0.78 () {56 vars in 1183 bytes} [Mon Sep 19 11:07:35 2022] GET /identity/v3/domains/dcd424585c2d46f4bc3e673c1ae14e71 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:35.409415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff8245-896c-4f5d-9f4a-16722ec9bddb None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:35.507329 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff8245-896c-4f5d-9f4a-16722ec9bddb None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:35.529760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:35.539933 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8a049-6522-42dc-836b-4d69124c72d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:35.540533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8a049-6522-42dc-836b-4d69124c72d8 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:35.540803 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8a049-6522-42dc-836b-4d69124c72d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:35.541071 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8a049-6522-42dc-836b-4d69124c72d8 None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:35.541705 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8a049-6522-42dc-836b-4d69124c72d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:35.543107 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8a049-6522-42dc-836b-4d69124c72d8 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:35.642591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 580/1177] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:07:35 2022] POST /identity/v3/users => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:07:37.849113 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9bd43-aaf6-4afc-baa1-9aa3deed9046 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:37.849689 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9bd43-aaf6-4afc-baa1-9aa3deed9046 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:37.849689 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9bd43-aaf6-4afc-baa1-9aa3deed9046 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:37.850971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 598/1178] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:07:37.856314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e900f40-1cb0-48e3-99ea-dd43da30bdb9 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:37.856760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e900f40-1cb0-48e3-99ea-dd43da30bdb9 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:37.856891 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e900f40-1cb0-48e3-99ea-dd43da30bdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:37.875112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-4e900f40-1cb0-48e3-99ea-dd43da30bdb9 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:07:37.876354 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4e900f40-1cb0-48e3-99ea-dd43da30bdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:07:37.947504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 581/1179] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:07:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:07:38.216874 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9bd43-aaf6-4afc-baa1-9aa3deed9046 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:38.267023 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:38.322264 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9bd43-aaf6-4afc-baa1-9aa3deed9046 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:38.356269 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bcc355-7540-4161-8833-572eaa2966b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:38.356896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcc355-7540-4161-8833-572eaa2966b6 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:38.357179 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcc355-7540-4161-8833-572eaa2966b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:38.357456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcc355-7540-4161-8833-572eaa2966b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:38.368759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bcc355-7540-4161-8833-572eaa2966b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:07:38.373924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-96bcc355-7540-4161-8833-572eaa2966b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:07:38.373924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:07:38.373924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:07:38.373924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:07:38.373924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:07:38.373924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:07:38.373924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:07:38.373924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:07:38.373924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:07:38.373924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:07:38.373924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:07:38.373924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:07:38.373924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:07:38.373924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:07:38.373924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:07:38.373924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:07:38.374953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:07:38.374953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:07:38.374953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:07:38.374953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:07:38.374953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:07:38.374953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:07:38.374953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:07:38.374953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:07:38.374953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:07:38.374953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:07:38.374953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:07:38.374953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:07:38.374953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:07:38.374953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:07:38.374953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:07:38.374953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:07:38.376216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:07:38.376216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:07:38.376216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:07:38.376216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:07:38.376216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:07:38.376216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:07:38.376216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:07:38.376216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:07:38.376216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:07:38.376216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:07:38.376216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:07:38.376216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:07:38.376216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 599/1180] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:07:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:07:38.381242 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e900f40-1cb0-48e3-99ea-dd43da30bdb9 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:38.479528 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e900f40-1cb0-48e3-99ea-dd43da30bdb9 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:38.490353 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:38.515768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9eb0b7-6d0f-4d9a-a1ea-a8e40e93b1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:38.516378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9eb0b7-6d0f-4d9a-a1ea-a8e40e93b1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:38.516745 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9eb0b7-6d0f-4d9a-a1ea-a8e40e93b1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:38.517018 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9eb0b7-6d0f-4d9a-a1ea-a8e40e93b1c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:38.517682 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9eb0b7-6d0f-4d9a-a1ea-a8e40e93b1c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:07:38.517983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9eb0b7-6d0f-4d9a-a1ea-a8e40e93b1c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:38.519902 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9eb0b7-6d0f-4d9a-a1ea-a8e40e93b1c0 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:38.528285 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 582/1181] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:07:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:38.534969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bcc355-7540-4161-8833-572eaa2966b6 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:38.635833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bcc355-7540-4161-8833-572eaa2966b6 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:38.637270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:38.669962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c25e5-34e3-4416-8556-9df4fd084afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:38.670585 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c25e5-34e3-4416-8556-9df4fd084afd None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:38.670862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c25e5-34e3-4416-8556-9df4fd084afd None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:38.671134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c25e5-34e3-4416-8556-9df4fd084afd None admin] PATH_INFO: `/v3/users/1e6e4e7625fa480fbfd1dc6b98a91937` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:38.683658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c25e5-34e3-4416-8556-9df4fd084afd None admin] RBAC: Authorizing `identity:get_user(user_id=1e6e4e7625fa480fbfd1dc6b98a91937)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:38.685079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c25e5-34e3-4416-8556-9df4fd084afd None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:38.686651 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 600/1182] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:07:38 2022] GET /identity/v3/users/1e6e4e7625fa480fbfd1dc6b98a91937 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:38.692549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9eb0b7-6d0f-4d9a-a1ea-a8e40e93b1c0 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:38.790815 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9eb0b7-6d0f-4d9a-a1ea-a8e40e93b1c0 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:38.823737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62831a-f976-4074-8694-663e2e424d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:38.824354 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62831a-f976-4074-8694-663e2e424d89 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:38.824633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62831a-f976-4074-8694-663e2e424d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:38.824907 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62831a-f976-4074-8694-663e2e424d89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:38.865267 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:38.894792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62831a-f976-4074-8694-663e2e424d89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:38.896248 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62831a-f976-4074-8694-663e2e424d89 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:38.991606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 583/1183] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:07:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:07:38.997645 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c25e5-34e3-4416-8556-9df4fd084afd None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:39.094301 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c25e5-34e3-4416-8556-9df4fd084afd None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:39.128983 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547bb40-9fc8-47bb-803e-4766d993c44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:39.129614 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547bb40-9fc8-47bb-803e-4766d993c44e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:39.129896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547bb40-9fc8-47bb-803e-4766d993c44e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:39.130176 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547bb40-9fc8-47bb-803e-4766d993c44e None admin] PATH_INFO: `/v3/projects/7f2b2f5434a1468499c4c755912eaeec` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:39.131325 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8547bb40-9fc8-47bb-803e-4766d993c44e None admin] RBAC: Authorizing `identity:get_project(project_id=7f2b2f5434a1468499c4c755912eaeec)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:39.132756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8547bb40-9fc8-47bb-803e-4766d993c44e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:39.134382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 601/1184] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:07:38 2022] GET /identity/v3/projects/7f2b2f5434a1468499c4c755912eaeec => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:39.140489 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62831a-f976-4074-8694-663e2e424d89 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:39.207778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:39.235590 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62831a-f976-4074-8694-663e2e424d89 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:39.270758 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f395ff-a0af-4fe3-9de4-326dd6d4d8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:39.271388 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f395ff-a0af-4fe3-9de4-326dd6d4d8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:39.273180 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f395ff-a0af-4fe3-9de4-326dd6d4d8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:39.273485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f395ff-a0af-4fe3-9de4-326dd6d4d8b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:39.274214 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f395ff-a0af-4fe3-9de4-326dd6d4d8b8 None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=7f2b2f5434a1468499c4c755912eaeec, user.id=1e6e4e7625fa480fbfd1dc6b98a91937) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:07:39.274523 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f395ff-a0af-4fe3-9de4-326dd6d4d8b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:39.277780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f395ff-a0af-4fe3-9de4-326dd6d4d8b8 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:39.283234 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 584/1185] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:07:39 2022] GET /identity/v3/role_assignments?user.id=1e6e4e7625fa480fbfd1dc6b98a91937&scope.project.id=7f2b2f5434a1468499c4c755912eaeec&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:41.490850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8310ac-2305-4fc7-96d4-21690b4cd121 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:41.491409 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8310ac-2305-4fc7-96d4-21690b4cd121 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:41.491409 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8310ac-2305-4fc7-96d4-21690b4cd121 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:41.492670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 602/1186] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:07:41.497977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8df79-0bed-43ab-aeff-74082ac258a5 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:41.498357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8df79-0bed-43ab-aeff-74082ac258a5 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:41.498518 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8df79-0bed-43ab-aeff-74082ac258a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:41.516204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-e5d8df79-0bed-43ab-aeff-74082ac258a5 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:07:41.517382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e5d8df79-0bed-43ab-aeff-74082ac258a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:07:41.583274 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 585/1187] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:07:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:07:41.849604 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8310ac-2305-4fc7-96d4-21690b4cd121 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:41.878592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:41.947922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8310ac-2305-4fc7-96d4-21690b4cd121 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:41.980716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bae298-686a-4744-b53b-0f53efe95dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:41.981330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bae298-686a-4744-b53b-0f53efe95dfa None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:41.981608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bae298-686a-4744-b53b-0f53efe95dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:41.981878 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bae298-686a-4744-b53b-0f53efe95dfa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:41.994707 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bae298-686a-4744-b53b-0f53efe95dfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:07:41.999783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-48bae298-686a-4744-b53b-0f53efe95dfa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:07:41.999783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:07:41.999783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:07:41.999783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:07:41.999783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:07:41.999783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:07:41.999783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:07:41.999783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:07:41.999783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:07:41.999783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:07:41.999783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:07:41.999783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:07:41.999783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:07:41.999783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:07:41.999783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:07:41.999783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:07:42.000806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:07:42.000806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:07:42.000806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:07:42.000806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:07:42.000806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:07:42.000806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:07:42.000806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:07:42.000806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:07:42.000806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:07:42.000806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:07:42.000806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:07:42.000806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:07:42.000806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:07:42.000806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:07:42.000806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:07:42.000806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:07:42.001855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:07:42.001855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:07:42.001855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:07:42.001855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:07:42.001855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:07:42.001855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:07:42.001855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:07:42.001855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:07:42.001855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:07:42.001855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:07:42.001855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:07:42.001855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:07:42.001855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 603/1188] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:07:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:07:42.007034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8df79-0bed-43ab-aeff-74082ac258a5 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:42.072067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:42.106726 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8df79-0bed-43ab-aeff-74082ac258a5 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:42.141373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc720436-ea72-4728-99f0-9f68b3bb63e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:42.141982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc720436-ea72-4728-99f0-9f68b3bb63e6 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:42.142265 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc720436-ea72-4728-99f0-9f68b3bb63e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:42.142535 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc720436-ea72-4728-99f0-9f68b3bb63e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:42.143221 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc720436-ea72-4728-99f0-9f68b3bb63e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:07:42.144062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc720436-ea72-4728-99f0-9f68b3bb63e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:42.145425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc720436-ea72-4728-99f0-9f68b3bb63e6 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:42.153370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 586/1189] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:07:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:42.159805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bae298-686a-4744-b53b-0f53efe95dfa None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:42.244995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:42.256765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bae298-686a-4744-b53b-0f53efe95dfa None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:42.290892 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66898913-7e49-4606-bd59-d0c187d8003a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:42.291546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66898913-7e49-4606-bd59-d0c187d8003a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:42.291834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66898913-7e49-4606-bd59-d0c187d8003a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:42.292111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66898913-7e49-4606-bd59-d0c187d8003a None admin] PATH_INFO: `/v3/users/1e6e4e7625fa480fbfd1dc6b98a91937` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:42.293227 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66898913-7e49-4606-bd59-d0c187d8003a None admin] RBAC: Authorizing `identity:get_user(user_id=1e6e4e7625fa480fbfd1dc6b98a91937)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:42.294658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66898913-7e49-4606-bd59-d0c187d8003a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:42.296377 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 604/1190] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:07:42 2022] GET /identity/v3/users/1e6e4e7625fa480fbfd1dc6b98a91937 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:42.302324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc720436-ea72-4728-99f0-9f68b3bb63e6 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:42.399925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc720436-ea72-4728-99f0-9f68b3bb63e6 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:42.425346 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:42.432756 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56d055-c60d-4f73-b0cc-c6c50dd156eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:42.433382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56d055-c60d-4f73-b0cc-c6c50dd156eb None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:42.433683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56d055-c60d-4f73-b0cc-c6c50dd156eb None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:42.433957 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56d055-c60d-4f73-b0cc-c6c50dd156eb None admin] PATH_INFO: `/v3/projects/7f2b2f5434a1468499c4c755912eaeec` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:42.435056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56d055-c60d-4f73-b0cc-c6c50dd156eb None admin] RBAC: Authorizing `identity:get_project(project_id=7f2b2f5434a1468499c4c755912eaeec)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:42.436502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56d055-c60d-4f73-b0cc-c6c50dd156eb None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:42.438200 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 587/1191] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:07:42 2022] GET /identity/v3/projects/7f2b2f5434a1468499c4c755912eaeec => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:42.444174 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66898913-7e49-4606-bd59-d0c187d8003a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:42.542633 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66898913-7e49-4606-bd59-d0c187d8003a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:42.575375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2bfa1-7f4e-4c15-8c76-f93f378d7638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:42.576034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2bfa1-7f4e-4c15-8c76-f93f378d7638 None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:42.576310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2bfa1-7f4e-4c15-8c76-f93f378d7638 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:42.576578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2bfa1-7f4e-4c15-8c76-f93f378d7638 None admin] PATH_INFO: `/v3/projects/7f2b2f5434a1468499c4c755912eaeec/users/1e6e4e7625fa480fbfd1dc6b98a91937/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:42.578452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2bfa1-7f4e-4c15-8c76-f93f378d7638 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f2b2f5434a1468499c4c755912eaeec, user_id=1e6e4e7625fa480fbfd1dc6b98a91937, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:42.579896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2bfa1-7f4e-4c15-8c76-f93f378d7638 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:42.651193 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 605/1192] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:07:42 2022] PUT /identity/v3/projects/7f2b2f5434a1468499c4c755912eaeec/users/1e6e4e7625fa480fbfd1dc6b98a91937/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:07:44.810292 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d431b86-e2ac-41e0-bf7a-593c8c60f06c None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:44.810854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d431b86-e2ac-41e0-bf7a-593c8c60f06c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:44.810854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d431b86-e2ac-41e0-bf7a-593c8c60f06c None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:44.812219 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 588/1193] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:07:44.817904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf7752-96ac-4bd7-bee5-5d59ec16204f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:44.818303 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf7752-96ac-4bd7-bee5-5d59ec16204f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:44.818456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf7752-96ac-4bd7-bee5-5d59ec16204f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:44.836835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-89bf7752-96ac-4bd7-bee5-5d59ec16204f None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:07:44.838025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-89bf7752-96ac-4bd7-bee5-5d59ec16204f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:07:44.848466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:44.903688 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 606/1194] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:07:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:07:45.191490 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d431b86-e2ac-41e0-bf7a-593c8c60f06c None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:45.291780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d431b86-e2ac-41e0-bf7a-593c8c60f06c None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:45.327729 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6eb14-4333-43ae-b2e2-857ef7e6318a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:45.328463 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6eb14-4333-43ae-b2e2-857ef7e6318a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:45.328743 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6eb14-4333-43ae-b2e2-857ef7e6318a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:45.329015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6eb14-4333-43ae-b2e2-857ef7e6318a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:45.339151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6eb14-4333-43ae-b2e2-857ef7e6318a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:07:45.344013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-2bf6eb14-4333-43ae-b2e2-857ef7e6318a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:07:45.344013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:07:45.344013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:07:45.344013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:07:45.344013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:07:45.344013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:07:45.344013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:07:45.344013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:07:45.344013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:07:45.344013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:07:45.344013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:07:45.344013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:07:45.344013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:07:45.344013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:07:45.344013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:07:45.344013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:07:45.345131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:07:45.345131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:07:45.345131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:07:45.345131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:07:45.345131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:07:45.345131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:07:45.345131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:07:45.345131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:07:45.345131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:07:45.345131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:07:45.345131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:07:45.345131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:07:45.345131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:07:45.345131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:07:45.345131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:07:45.345131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:07:45.346336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:07:45.346336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:07:45.346336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:07:45.346336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:07:45.346336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:07:45.346336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:07:45.346336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:07:45.346336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:07:45.346336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:07:45.346336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:07:45.346336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:07:45.346336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:07:45.346336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 589/1195] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:07:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:07:45.351277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf7752-96ac-4bd7-bee5-5d59ec16204f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:45.451931 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf7752-96ac-4bd7-bee5-5d59ec16204f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:45.484219 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48f1d0-83f6-403c-a8ee-3a294dcb0295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:45.484827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48f1d0-83f6-403c-a8ee-3a294dcb0295 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:45.485110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48f1d0-83f6-403c-a8ee-3a294dcb0295 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:45.485385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48f1d0-83f6-403c-a8ee-3a294dcb0295 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:45.486047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48f1d0-83f6-403c-a8ee-3a294dcb0295 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:07:45.486352 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48f1d0-83f6-403c-a8ee-3a294dcb0295 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:45.488270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48f1d0-83f6-403c-a8ee-3a294dcb0295 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:45.496209 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 607/1196] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:07:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:45.502456 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6eb14-4333-43ae-b2e2-857ef7e6318a None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:45.558356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:45.601249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6eb14-4333-43ae-b2e2-857ef7e6318a None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:45.635396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af954b9-6b97-4a5d-ab89-b9e17e11dddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:45.636028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af954b9-6b97-4a5d-ab89-b9e17e11dddc None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:45.636309 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af954b9-6b97-4a5d-ab89-b9e17e11dddc None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:45.636585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af954b9-6b97-4a5d-ab89-b9e17e11dddc None admin] PATH_INFO: `/v3/users/1e6e4e7625fa480fbfd1dc6b98a91937` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:45.637701 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af954b9-6b97-4a5d-ab89-b9e17e11dddc None admin] RBAC: Authorizing `identity:get_user(user_id=1e6e4e7625fa480fbfd1dc6b98a91937)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:45.639125 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af954b9-6b97-4a5d-ab89-b9e17e11dddc None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:45.640728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 590/1197] 10.222.0.78 () {56 vars in 1177 bytes} [Mon Sep 19 11:07:45 2022] GET /identity/v3/users/1e6e4e7625fa480fbfd1dc6b98a91937 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:45.646561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48f1d0-83f6-403c-a8ee-3a294dcb0295 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:45.658257 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:45.745408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48f1d0-83f6-403c-a8ee-3a294dcb0295 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:45.777790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fcf18-d17b-42a8-9806-e9cb026885e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:45.778388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fcf18-d17b-42a8-9806-e9cb026885e7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:45.778671 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fcf18-d17b-42a8-9806-e9cb026885e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:45.778950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fcf18-d17b-42a8-9806-e9cb026885e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:45.843516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fcf18-d17b-42a8-9806-e9cb026885e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:45.844957 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fcf18-d17b-42a8-9806-e9cb026885e7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:45.854680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:45.940242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 608/1198] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:07:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 11:07:45.946205 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af954b9-6b97-4a5d-ab89-b9e17e11dddc None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:46.043148 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af954b9-6b97-4a5d-ab89-b9e17e11dddc None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:46.065497 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:46.075707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf334f3-bfaf-466e-86b2-1f04afdac4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:46.076335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf334f3-bfaf-466e-86b2-1f04afdac4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:46.076614 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf334f3-bfaf-466e-86b2-1f04afdac4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:46.076925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf334f3-bfaf-466e-86b2-1f04afdac4e0 None admin] PATH_INFO: `/v3/projects/7f2b2f5434a1468499c4c755912eaeec` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:46.078015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf334f3-bfaf-466e-86b2-1f04afdac4e0 None admin] RBAC: Authorizing `identity:get_project(project_id=7f2b2f5434a1468499c4c755912eaeec)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:46.079469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf334f3-bfaf-466e-86b2-1f04afdac4e0 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:46.081646 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 591/1199] 10.222.0.78 () {56 vars in 1186 bytes} [Mon Sep 19 11:07:45 2022] GET /identity/v3/projects/7f2b2f5434a1468499c4c755912eaeec => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:46.087858 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fcf18-d17b-42a8-9806-e9cb026885e7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:07:46.184658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fcf18-d17b-42a8-9806-e9cb026885e7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:07:46.193622 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:46.217552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d10432-8bff-4c89-b401-aebab8529393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:07:46.218153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d10432-8bff-4c89-b401-aebab8529393 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:07:46.218438 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d10432-8bff-4c89-b401-aebab8529393 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:07:46.218712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d10432-8bff-4c89-b401-aebab8529393 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:07:46.219465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d10432-8bff-4c89-b401-aebab8529393 None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=7f2b2f5434a1468499c4c755912eaeec, user.id=1e6e4e7625fa480fbfd1dc6b98a91937) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:07:46.219777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d10432-8bff-4c89-b401-aebab8529393 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:07:46.222077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d10432-8bff-4c89-b401-aebab8529393 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:07:46.227993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 609/1200] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:07:46 2022] GET /identity/v3/role_assignments?user.id=1e6e4e7625fa480fbfd1dc6b98a91937&scope.project.id=7f2b2f5434a1468499c4c755912eaeec&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:07:56.110733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:07:56.225385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=14 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:01.328146 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa04b7-493d-4b5d-adb5-ae4c25054aff None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:01.328731 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa04b7-493d-4b5d-adb5-ae4c25054aff None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:01.328731 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa04b7-493d-4b5d-adb5-ae4c25054aff None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:01.330387 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 592/1201] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:01.335786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d2fad-7daa-43af-9da8-a2fadb480ff8 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:01.345078 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d2fad-7daa-43af-9da8-a2fadb480ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:01.345078 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d2fad-7daa-43af-9da8-a2fadb480ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:01.354544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e64d2fad-7daa-43af-9da8-a2fadb480ff8 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:01.359915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e64d2fad-7daa-43af-9da8-a2fadb480ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:01.427580 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 610/1202] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:09:01.700375 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa04b7-493d-4b5d-adb5-ae4c25054aff None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:01.805178 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa04b7-493d-4b5d-adb5-ae4c25054aff None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:01.840691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9146faa-4691-43dd-9c3d-94b8949dfedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:01.841326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9146faa-4691-43dd-9c3d-94b8949dfedb None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:01.841603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9146faa-4691-43dd-9c3d-94b8949dfedb None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:01.841871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9146faa-4691-43dd-9c3d-94b8949dfedb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:01.852512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9146faa-4691-43dd-9c3d-94b8949dfedb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:01.854180 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9146faa-4691-43dd-9c3d-94b8949dfedb None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:01.868400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-f9146faa-4691-43dd-9c3d-94b8949dfedb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:01.868400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:01.868400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:01.868400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:01.868400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:01.868400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:01.868400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:01.868400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:01.868400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:01.868400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:01.868400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:01.868400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:01.868400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:09:01.868400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:09:01.868400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:09:01.868400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:09:01.869757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:01.869757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:01.869757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:01.869757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:01.869757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:01.869757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:01.869757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:01.869757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:01.869757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:01.869757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:01.869757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:01.869757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:01.869757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:01.869757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:01.869757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:01.869757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:01.869757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:09:01.870950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:09:01.870950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:01.870950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:09:01.870950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 593/1203] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:01.876180 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d2fad-7daa-43af-9da8-a2fadb480ff8 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:01.973477 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d2fad-7daa-43af-9da8-a2fadb480ff8 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:02.001708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:02.007453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdfb78-786e-4a4d-a518-56a8c0300906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:02.008067 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdfb78-786e-4a4d-a518-56a8c0300906 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:02.008346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdfb78-786e-4a4d-a518-56a8c0300906 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:02.008617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdfb78-786e-4a4d-a518-56a8c0300906 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:02.009263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdfb78-786e-4a4d-a518-56a8c0300906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:02.009557 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdfb78-786e-4a4d-a518-56a8c0300906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:02.010917 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdfb78-786e-4a4d-a518-56a8c0300906 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:02.021920 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 611/1204] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:02.028213 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9146faa-4691-43dd-9c3d-94b8949dfedb None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:02.081122 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:02.128279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9146faa-4691-43dd-9c3d-94b8949dfedb None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:02.162623 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1c104-1b9c-4ff4-9197-b2f513ca3a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:02.163267 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1c104-1b9c-4ff4-9197-b2f513ca3a59 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:02.164168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1c104-1b9c-4ff4-9197-b2f513ca3a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:02.164461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1c104-1b9c-4ff4-9197-b2f513ca3a59 None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:02.165090 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab1c104-1b9c-4ff4-9197-b2f513ca3a59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:02.166514 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab1c104-1b9c-4ff4-9197-b2f513ca3a59 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:02.275858 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 594/1205] 10.222.0.78 () {60 vars in 1132 bytes} [Mon Sep 19 11:09:02 2022] POST /identity/v3/users => generated 251 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:09:04.489343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf6832-2494-4e13-84eb-be31f05cd7a3 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:04.489855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf6832-2494-4e13-84eb-be31f05cd7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:04.489855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf6832-2494-4e13-84eb-be31f05cd7a3 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:04.491154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 612/1206] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:04.496660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23acd2-1a27-4382-abb9-050ac66d07ac None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:04.497122 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23acd2-1a27-4382-abb9-050ac66d07ac None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:04.497220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23acd2-1a27-4382-abb9-050ac66d07ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:04.515723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-7e23acd2-1a27-4382-abb9-050ac66d07ac None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:04.516909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7e23acd2-1a27-4382-abb9-050ac66d07ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:04.582715 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 595/1207] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:09:04.856421 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf6832-2494-4e13-84eb-be31f05cd7a3 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:04.960028 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf6832-2494-4e13-84eb-be31f05cd7a3 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:04.993229 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec395d2-ce45-4788-b4f7-763c275231a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:04.993841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec395d2-ce45-4788-b4f7-763c275231a3 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:04.994201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec395d2-ce45-4788-b4f7-763c275231a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:04.994701 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec395d2-ce45-4788-b4f7-763c275231a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:05.004820 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec395d2-ce45-4788-b4f7-763c275231a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:05.011607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-dec395d2-ce45-4788-b4f7-763c275231a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:09:05.011607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:05.011607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:05.011607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:05.011607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:05.011607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:05.011607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:05.011607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:05.011607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:05.011607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:05.011607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:05.011607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:05.011607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:05.011607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:05.011607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:05.011607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:05.017490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:05.017490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:05.017490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:05.017490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:05.017490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:05.017490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:05.017490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:05.017490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:05.017490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:05.017490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:05.017490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:05.017490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:05.017490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:05.017490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:05.017490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:05.017490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:05.018634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:05.018634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:05.018634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:05.018634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:05.018634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:05.018634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:05.018634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:05.018634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:05.018634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:05.018634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:05.018634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:09:05.018634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:05.018634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 613/1208] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:09:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:05.019403 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e23acd2-1a27-4382-abb9-050ac66d07ac None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:05.077111 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:05.136160 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e23acd2-1a27-4382-abb9-050ac66d07ac None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:05.175910 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3e4a3-02c7-4c97-9e2a-579991aa9fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:05.176559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3e4a3-02c7-4c97-9e2a-579991aa9fab None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:05.176810 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3e4a3-02c7-4c97-9e2a-579991aa9fab None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:05.177196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3e4a3-02c7-4c97-9e2a-579991aa9fab None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:05.177822 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3e4a3-02c7-4c97-9e2a-579991aa9fab None admin] RBAC: Adding query filter params (name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:05.178131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3e4a3-02c7-4c97-9e2a-579991aa9fab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:05.180066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3e4a3-02c7-4c97-9e2a-579991aa9fab None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:05.188916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 596/1209] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:09:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:05.195807 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec395d2-ce45-4788-b4f7-763c275231a3 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:05.261881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:05.319082 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec395d2-ce45-4788-b4f7-763c275231a3 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:05.358800 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b5f7d-2f1e-4a08-acff-e684ab00adac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:05.359470 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b5f7d-2f1e-4a08-acff-e684ab00adac None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:05.359754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b5f7d-2f1e-4a08-acff-e684ab00adac None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:05.360022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b5f7d-2f1e-4a08-acff-e684ab00adac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:05.374940 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b5f7d-2f1e-4a08-acff-e684ab00adac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:05.374940 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b5f7d-2f1e-4a08-acff-e684ab00adac None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:05.388490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-de2b5f7d-2f1e-4a08-acff-e684ab00adac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:05.388490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:05.388490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:05.388490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:05.388490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:05.388490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:05.388490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:05.388490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:05.388490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:05.388490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:05.388490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:05.388490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:05.388490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:09:05.388490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:09:05.388490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:09:05.388490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:09:05.392366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:05.392366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:05.392366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:05.392366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:05.392366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:05.392366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:05.392366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:05.392366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:05.392366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:05.392366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:05.392366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:05.392366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:05.392366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:05.392366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:05.392366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:05.392366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:05.392366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:09:05.393581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:09:05.393581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:05.393581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:05.393581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 614/1210] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:05.396184 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3e4a3-02c7-4c97-9e2a-579991aa9fab None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:05.522208 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:05.525278 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3e4a3-02c7-4c97-9e2a-579991aa9fab None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:05.565790 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698b5c1-9d74-4031-b770-f65883c8e322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:05.566832 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698b5c1-9d74-4031-b770-f65883c8e322 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:05.566832 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698b5c1-9d74-4031-b770-f65883c8e322 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:05.567028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698b5c1-9d74-4031-b770-f65883c8e322 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:05.567629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698b5c1-9d74-4031-b770-f65883c8e322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:05.567922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698b5c1-9d74-4031-b770-f65883c8e322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:05.570326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698b5c1-9d74-4031-b770-f65883c8e322 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:05.580826 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 597/1211] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:05.587230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b5f7d-2f1e-4a08-acff-e684ab00adac None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:05.698727 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b5f7d-2f1e-4a08-acff-e684ab00adac None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:05.700312 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:05.741146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286aa9d-b2de-4045-84ca-5b41d44a9a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:05.741146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286aa9d-b2de-4045-84ca-5b41d44a9a29 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:05.741146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286aa9d-b2de-4045-84ca-5b41d44a9a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:05.741146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286aa9d-b2de-4045-84ca-5b41d44a9a29 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:05.746876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286aa9d-b2de-4045-84ca-5b41d44a9a29 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:05.748358 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286aa9d-b2de-4045-84ca-5b41d44a9a29 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:05.760237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-7286aa9d-b2de-4045-84ca-5b41d44a9a29 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:09:05.760237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:05.760237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:05.760237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:05.760237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:05.760237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:05.760237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:05.760237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:05.760237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:05.760237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:05.760237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:05.760237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:05.760237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:09:05.760237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:09:05.760237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:09:05.760237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:05.762975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:05.764124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:05.764124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:05.764124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:09:05.764124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:09:05.764124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:09:05.764124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:09:05.764124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:09:05.764124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:09:05.764124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:09:05.764124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:05.764124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 615/1212] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:05.768035 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698b5c1-9d74-4031-b770-f65883c8e322 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:05.883240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698b5c1-9d74-4031-b770-f65883c8e322 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:05.924811 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ad21e-5efe-43a6-a6f9-0b39e9877cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:05.925468 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ad21e-5efe-43a6-a6f9-0b39e9877cbf None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:05.925772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ad21e-5efe-43a6-a6f9-0b39e9877cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:05.926042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ad21e-5efe-43a6-a6f9-0b39e9877cbf None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:05.926768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ad21e-5efe-43a6-a6f9-0b39e9877cbf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:05.927068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ad21e-5efe-43a6-a6f9-0b39e9877cbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:05.929772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ad21e-5efe-43a6-a6f9-0b39e9877cbf None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:05.947542 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 598/1213] 10.222.0.78 () {56 vars in 1143 bytes} [Mon Sep 19 11:09:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:05.958540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286aa9d-b2de-4045-84ca-5b41d44a9a29 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:06.055039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286aa9d-b2de-4045-84ca-5b41d44a9a29 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:06.088605 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49df31b-2db6-4859-97ff-7ffe828756e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:06.089214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49df31b-2db6-4859-97ff-7ffe828756e9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:06.089496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49df31b-2db6-4859-97ff-7ffe828756e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:06.089769 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49df31b-2db6-4859-97ff-7ffe828756e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:06.120027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:06.158235 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49df31b-2db6-4859-97ff-7ffe828756e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:06.159808 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49df31b-2db6-4859-97ff-7ffe828756e9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:06.255546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 616/1214] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:09:06.261751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ad21e-5efe-43a6-a6f9-0b39e9877cbf None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:06.280805 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:06.362976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ad21e-5efe-43a6-a6f9-0b39e9877cbf None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:06.397704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464c70b-ba88-49fc-a679-78f25aea2500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:06.398362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464c70b-ba88-49fc-a679-78f25aea2500 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:06.398644 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464c70b-ba88-49fc-a679-78f25aea2500 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:06.398934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464c70b-ba88-49fc-a679-78f25aea2500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:06.409640 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b464c70b-ba88-49fc-a679-78f25aea2500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:06.411089 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b464c70b-ba88-49fc-a679-78f25aea2500 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:06.426892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-b464c70b-ba88-49fc-a679-78f25aea2500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:06.426892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:06.426892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:06.426892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:06.426892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:06.426892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:06.426892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:06.426892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:06.426892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:06.426892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:06.426892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:06.426892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:06.426892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:09:06.426892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:09:06.426892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:09:06.426892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:09:06.431042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:06.431042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:06.431042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:06.431042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:06.431042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:06.431042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:06.431042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:06.431042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:06.431042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:06.431042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:06.431042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:06.431042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:06.431042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:06.431042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:06.431042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:06.431042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:06.431042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:09:06.432237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:09:06.432237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:06.432237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:09:06.432237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 599/1215] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:06.434668 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49df31b-2db6-4859-97ff-7ffe828756e9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:06.496623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:06.534288 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49df31b-2db6-4859-97ff-7ffe828756e9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:06.567839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d599f57-d725-47b9-adf8-fc4de7209445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:06.568450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d599f57-d725-47b9-adf8-fc4de7209445 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:06.568732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d599f57-d725-47b9-adf8-fc4de7209445 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:06.569006 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d599f57-d725-47b9-adf8-fc4de7209445 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:06.570209 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d599f57-d725-47b9-adf8-fc4de7209445 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:06.570516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d599f57-d725-47b9-adf8-fc4de7209445 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:06.571974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d599f57-d725-47b9-adf8-fc4de7209445 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:06.584015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 617/1216] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:06.589954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464c70b-ba88-49fc-a679-78f25aea2500 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:06.662828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:06.693872 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464c70b-ba88-49fc-a679-78f25aea2500 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:06.728758 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b09c3-36ef-40a7-aae7-ba7813536179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:06.729391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b09c3-36ef-40a7-aae7-ba7813536179 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:06.729672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b09c3-36ef-40a7-aae7-ba7813536179 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:06.729945 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b09c3-36ef-40a7-aae7-ba7813536179 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:06.736802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b09c3-36ef-40a7-aae7-ba7813536179 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:06.738247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b09c3-36ef-40a7-aae7-ba7813536179 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:06.750449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-528b09c3-36ef-40a7-aae7-ba7813536179 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:09:06.750449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:06.750449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:06.750449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:06.750449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:06.750449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:06.750449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:06.750449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:06.750449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:06.750449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:06.750449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:06.750449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:06.750449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:09:06.750449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:09:06.750449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:09:06.752989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:09:06.752989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:06.752989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:06.752989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:06.752989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:06.752989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:06.752989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:06.752989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:06.752989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:06.752989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:06.752989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:06.752989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:06.752989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:06.752989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:06.752989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:06.752989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:06.752989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:06.754054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:09:06.754054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:09:06.754054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:09:06.754054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:09:06.754054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:09:06.754054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:09:06.754054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:09:06.754054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:09:06.754054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 600/1217] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:09:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:06.758669 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d599f57-d725-47b9-adf8-fc4de7209445 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:06.858501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d599f57-d725-47b9-adf8-fc4de7209445 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:06.880422 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:06.892093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cd6de-22c9-4471-921c-03259816e92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:06.892693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cd6de-22c9-4471-921c-03259816e92e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:06.892975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cd6de-22c9-4471-921c-03259816e92e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:06.893245 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cd6de-22c9-4471-921c-03259816e92e None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:06.893924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652cd6de-22c9-4471-921c-03259816e92e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:06.894230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652cd6de-22c9-4471-921c-03259816e92e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:06.895661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652cd6de-22c9-4471-921c-03259816e92e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:06.907342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 618/1218] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:09:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:06.914215 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b09c3-36ef-40a7-aae7-ba7813536179 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:07.022534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b09c3-36ef-40a7-aae7-ba7813536179 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:07.051247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:07.063100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5f59e-0afe-4af1-a2ee-fd74b558a0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:07.063562 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5f59e-0afe-4af1-a2ee-fd74b558a0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:07.064061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5f59e-0afe-4af1-a2ee-fd74b558a0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:07.064161 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5f59e-0afe-4af1-a2ee-fd74b558a0d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:07.064828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5f59e-0afe-4af1-a2ee-fd74b558a0d9 None admin] RBAC: Adding query filter params (role.id=28365abffd3d4ff6874bce2e0c561f1d, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=0427cdd3905a422cab4fed0aac187ce2) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:07.065133 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5f59e-0afe-4af1-a2ee-fd74b558a0d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:07.066575 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5f59e-0afe-4af1-a2ee-fd74b558a0d9 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:07.073319 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 601/1219] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:06 2022] GET /identity/v3/role_assignments?user.id=0427cdd3905a422cab4fed0aac187ce2&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=28365abffd3d4ff6874bce2e0c561f1d => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:09.230643 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaa6b5-b66e-4d9b-963e-96ef3789bb7b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:09.231219 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaa6b5-b66e-4d9b-963e-96ef3789bb7b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:09.231219 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaa6b5-b66e-4d9b-963e-96ef3789bb7b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:09.232592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 619/1220] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:09.237961 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130facd-bd16-461a-b5d1-05ae33df6af8 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:09.238369 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130facd-bd16-461a-b5d1-05ae33df6af8 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:09.238515 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130facd-bd16-461a-b5d1-05ae33df6af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:09.257469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-3130facd-bd16-461a-b5d1-05ae33df6af8 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:09.262800 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-3130facd-bd16-461a-b5d1-05ae33df6af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:09.326776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 602/1221] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:09:09.595238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aaa6b5-b66e-4d9b-963e-96ef3789bb7b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:09.697383 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aaa6b5-b66e-4d9b-963e-96ef3789bb7b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:09.731181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d7a0c-56d0-41a3-8298-8125c9c3fbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:09.731822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d7a0c-56d0-41a3-8298-8125c9c3fbed None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:09.732102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d7a0c-56d0-41a3-8298-8125c9c3fbed None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:09.732416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d7a0c-56d0-41a3-8298-8125c9c3fbed None admin] PATH_INFO: `/v3/roles/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:09.742656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d7a0c-56d0-41a3-8298-8125c9c3fbed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:09.748019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-e26d7a0c-56d0-41a3-8298-8125c9c3fbed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:09:09.748019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:09.748019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:09.748019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:09.748019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:09.748019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:09.748019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:09.748019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:09.748019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:09.748019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:09.748019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:09.748019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:09.748019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:09.748019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:09.748019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:09.748019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:09.764005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:09.764005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:09.764005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:09.764005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:09.764005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:09.764005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:09.764005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:09.764005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:09.764005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:09.764005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:09.764005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:09.764005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:09.764005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:09.764005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:09.764005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:09.764005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:09.764998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:09.764998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:09.764998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:09.764998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:09.764998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:09.764998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:09.764998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:09.764998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:09.764998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:09.764998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:09.764998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:09:09.764998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:09.764998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 620/1222] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:09:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:09.765860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3130facd-bd16-461a-b5d1-05ae33df6af8 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:09.855644 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3130facd-bd16-461a-b5d1-05ae33df6af8 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:09.878301 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:09.889428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9e075-5132-43ac-bc3c-a6ceb92ddd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:09.890041 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9e075-5132-43ac-bc3c-a6ceb92ddd32 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:09.890316 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9e075-5132-43ac-bc3c-a6ceb92ddd32 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:09.890591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9e075-5132-43ac-bc3c-a6ceb92ddd32 None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:09.891274 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9e075-5132-43ac-bc3c-a6ceb92ddd32 None admin] RBAC: Adding query filter params (name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:09.891579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9e075-5132-43ac-bc3c-a6ceb92ddd32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:09.892941 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9e075-5132-43ac-bc3c-a6ceb92ddd32 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:09.901329 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 603/1223] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:09:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:09.906854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d7a0c-56d0-41a3-8298-8125c9c3fbed None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:09.950540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:10.003352 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d7a0c-56d0-41a3-8298-8125c9c3fbed None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:10.036114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5124d-8ea3-4322-9faf-f862e8d72427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:10.036736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5124d-8ea3-4322-9faf-f862e8d72427 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:10.037016 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5124d-8ea3-4322-9faf-f862e8d72427 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:10.037289 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5124d-8ea3-4322-9faf-f862e8d72427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:10.046994 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5124d-8ea3-4322-9faf-f862e8d72427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:10.048484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5124d-8ea3-4322-9faf-f862e8d72427 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:10.062525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-aff5124d-8ea3-4322-9faf-f862e8d72427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:10.062525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:10.062525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:10.062525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:10.062525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:10.062525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:10.062525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:10.062525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:10.062525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:10.062525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:10.062525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:10.062525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:10.062525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:09:10.062525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:09:10.062525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:09:10.062525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:09:10.063697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:10.063697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:10.063697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:10.063697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:10.063697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:10.063697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:10.063697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:10.063697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:10.063697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:10.063697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:10.063697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:10.063697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:10.063697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:10.063697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:10.063697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:10.063697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:10.063697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:09:10.065000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:09:10.065000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:10.065000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:10.065000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 621/1224] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:10.070155 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9e075-5132-43ac-bc3c-a6ceb92ddd32 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:10.180483 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9e075-5132-43ac-bc3c-a6ceb92ddd32 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:10.214091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45e185-0174-44d0-b0e4-f0712949dade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:10.214698 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45e185-0174-44d0-b0e4-f0712949dade None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:10.214979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45e185-0174-44d0-b0e4-f0712949dade None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:10.215291 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45e185-0174-44d0-b0e4-f0712949dade None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:10.215936 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45e185-0174-44d0-b0e4-f0712949dade None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:10.216297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45e185-0174-44d0-b0e4-f0712949dade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:10.217714 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45e185-0174-44d0-b0e4-f0712949dade None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:10.229497 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 604/1225] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:10.235310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5124d-8ea3-4322-9faf-f862e8d72427 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:10.327810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:10.334297 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5124d-8ea3-4322-9faf-f862e8d72427 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:10.368444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbab7f-0c11-4e03-955c-76759b85e638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:10.369000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbab7f-0c11-4e03-955c-76759b85e638 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:10.369529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbab7f-0c11-4e03-955c-76759b85e638 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:10.369804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbab7f-0c11-4e03-955c-76759b85e638 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:10.376734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefbab7f-0c11-4e03-955c-76759b85e638 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:10.378166 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefbab7f-0c11-4e03-955c-76759b85e638 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:10.389985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-fefbab7f-0c11-4e03-955c-76759b85e638 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:09:10.389985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:10.389985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:10.389985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:10.389985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:10.389985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:10.389985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:10.389985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:10.389985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:10.389985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:10.389985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:10.389985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:10.389985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:09:10.389985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:09:10.389985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:09:10.389985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:10.391114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:10.392356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:10.392356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:10.392356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:09:10.392356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:09:10.392356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:09:10.392356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:09:10.392356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:09:10.392356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:09:10.392356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:09:10.392356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:10.392356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 622/1226] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:10.397077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45e185-0174-44d0-b0e4-f0712949dade None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:10.448444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:10.500405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45e185-0174-44d0-b0e4-f0712949dade None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:10.532686 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fabfd7-0859-4aca-98fc-dfd14b88d8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:10.533283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fabfd7-0859-4aca-98fc-dfd14b88d8ac None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:10.533564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fabfd7-0859-4aca-98fc-dfd14b88d8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:10.533837 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fabfd7-0859-4aca-98fc-dfd14b88d8ac None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:10.534559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fabfd7-0859-4aca-98fc-dfd14b88d8ac None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:10.534862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fabfd7-0859-4aca-98fc-dfd14b88d8ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:10.536307 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fabfd7-0859-4aca-98fc-dfd14b88d8ac None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:10.554724 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 605/1227] 10.222.0.78 () {56 vars in 1143 bytes} [Mon Sep 19 11:09:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:10.560710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbab7f-0c11-4e03-955c-76759b85e638 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:10.672134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbab7f-0c11-4e03-955c-76759b85e638 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:10.707903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127562a-de91-44c4-a553-342dd29fb272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:10.708545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127562a-de91-44c4-a553-342dd29fb272 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:10.708829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127562a-de91-44c4-a553-342dd29fb272 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:10.709105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127562a-de91-44c4-a553-342dd29fb272 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:10.720346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127562a-de91-44c4-a553-342dd29fb272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:10.721817 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127562a-de91-44c4-a553-342dd29fb272 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:10.736521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-a127562a-de91-44c4-a553-342dd29fb272 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:10.736521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:10.736521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:10.736521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:10.736521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:10.736521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:10.736521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:10.736521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:10.736521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:10.736521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:10.736521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:10.736521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:10.736521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:09:10.736521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:09:10.736521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:09:10.736521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:09:10.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:10.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:10.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:10.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:10.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:10.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:10.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:10.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:10.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:10.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:10.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:10.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:10.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:10.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:10.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:10.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:10.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:09:10.738571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:09:10.738571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:10.738571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:10.738571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 623/1228] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:10.743631 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fabfd7-0859-4aca-98fc-dfd14b88d8ac None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:10.863007 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fabfd7-0859-4aca-98fc-dfd14b88d8ac None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:10.865318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:10.896988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b981210-f27a-447b-8b12-0ba0eaf7ef59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:10.897615 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b981210-f27a-447b-8b12-0ba0eaf7ef59 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:10.897924 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b981210-f27a-447b-8b12-0ba0eaf7ef59 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:10.898168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b981210-f27a-447b-8b12-0ba0eaf7ef59 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:10.898808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b981210-f27a-447b-8b12-0ba0eaf7ef59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:10.899115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b981210-f27a-447b-8b12-0ba0eaf7ef59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:10.900530 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b981210-f27a-447b-8b12-0ba0eaf7ef59 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:10.911631 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 606/1229] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:10.917653 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127562a-de91-44c4-a553-342dd29fb272 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:10.926983 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:11.026336 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127562a-de91-44c4-a553-342dd29fb272 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:11.059114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef4aa0-2037-4051-ad00-3f6eff0b88ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:11.059772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef4aa0-2037-4051-ad00-3f6eff0b88ae None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:11.060053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef4aa0-2037-4051-ad00-3f6eff0b88ae None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:11.060325 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef4aa0-2037-4051-ad00-3f6eff0b88ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:11.066971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef4aa0-2037-4051-ad00-3f6eff0b88ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:11.068527 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef4aa0-2037-4051-ad00-3f6eff0b88ae None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:11.087061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-1aef4aa0-2037-4051-ad00-3f6eff0b88ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:09:11.087061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:11.087061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:11.087061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:11.087061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:11.087061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:11.087061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:11.087061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:11.087061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:11.087061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:11.087061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:11.087061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:11.087061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:09:11.087061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:09:11.087061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:09:11.088058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:09:11.088058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:11.088058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:11.088058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:11.088058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:11.088058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:11.088058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:11.088058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:11.088058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:11.088058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:11.088058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:11.088058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:11.088058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:11.088058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:11.088058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:11.088058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:11.088058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:11.088967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:09:11.088967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:09:11.088967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:09:11.088967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:09:11.088967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:09:11.088967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:09:11.088967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:09:11.088967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:11.089467 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 624/1230] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:09:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:11.095372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b981210-f27a-447b-8b12-0ba0eaf7ef59 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:11.202142 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b981210-f27a-447b-8b12-0ba0eaf7ef59 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:11.236199 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb573a1-8fbf-4001-8638-8b5388adab52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:11.236831 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb573a1-8fbf-4001-8638-8b5388adab52 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:11.237106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb573a1-8fbf-4001-8638-8b5388adab52 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:11.237377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb573a1-8fbf-4001-8638-8b5388adab52 None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:11.238062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb573a1-8fbf-4001-8638-8b5388adab52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:11.238363 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb573a1-8fbf-4001-8638-8b5388adab52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:11.239808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb573a1-8fbf-4001-8638-8b5388adab52 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:11.252880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 607/1231] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:09:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:11.259444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef4aa0-2037-4051-ad00-3f6eff0b88ae None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:11.325519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:11.360184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef4aa0-2037-4051-ad00-3f6eff0b88ae None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:11.394716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd4cca-d578-4cfe-9ecf-bb96b6ea84fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:11.395413 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd4cca-d578-4cfe-9ecf-bb96b6ea84fc None admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:11.395694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd4cca-d578-4cfe-9ecf-bb96b6ea84fc None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:11.395964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd4cca-d578-4cfe-9ecf-bb96b6ea84fc None admin] PATH_INFO: `/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/0427cdd3905a422cab4fed0aac187ce2/roles/28365abffd3d4ff6874bce2e0c561f1d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:11.410922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd4cca-d578-4cfe-9ecf-bb96b6ea84fc None admin] RBAC: Authorizing `identity:create_grant(project_id=a8563e86b7c94ff29e39fb20a7969551, user_id=0427cdd3905a422cab4fed0aac187ce2, role_id=28365abffd3d4ff6874bce2e0c561f1d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:11.412397 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd4cca-d578-4cfe-9ecf-bb96b6ea84fc None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:11.519897 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 625/1232] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:09:11 2022] PUT /identity/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/0427cdd3905a422cab4fed0aac187ce2/roles/28365abffd3d4ff6874bce2e0c561f1d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:09:13.693392 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8481816-145e-4ae2-9fba-d1c873a63e44 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:13.693960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8481816-145e-4ae2-9fba-d1c873a63e44 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:13.693960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8481816-145e-4ae2-9fba-d1c873a63e44 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:13.696103 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 608/1233] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:09:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:13.701331 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8d802-3ba2-4603-aa3b-5713a0f8f38a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:13.701744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8d802-3ba2-4603-aa3b-5713a0f8f38a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:13.701891 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8d802-3ba2-4603-aa3b-5713a0f8f38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:13.721085 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-cab8d802-3ba2-4603-aa3b-5713a0f8f38a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:13.722282 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-cab8d802-3ba2-4603-aa3b-5713a0f8f38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:13.789222 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 626/1234] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:09:14.064891 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8481816-145e-4ae2-9fba-d1c873a63e44 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:14.090376 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:14.167697 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8481816-145e-4ae2-9fba-d1c873a63e44 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:14.202334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09c025-4053-4c9b-af80-4a3b7be2425e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:14.207695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09c025-4053-4c9b-af80-4a3b7be2425e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:14.207695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09c025-4053-4c9b-af80-4a3b7be2425e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:14.207695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09c025-4053-4c9b-af80-4a3b7be2425e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:14.213731 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc09c025-4053-4c9b-af80-4a3b7be2425e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:14.218740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-cc09c025-4053-4c9b-af80-4a3b7be2425e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:09:14.218740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:14.218740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:14.218740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:14.218740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:14.218740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:14.218740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:14.218740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:14.218740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:14.218740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:14.218740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:14.218740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:14.218740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:14.218740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:14.218740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:14.218740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:14.219894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:14.219894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:14.219894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:14.219894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:14.219894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:14.219894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:14.219894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:14.219894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:14.219894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:14.219894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:14.219894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:14.219894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:14.219894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:14.219894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:14.219894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:14.219894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:14.220896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:14.220896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:14.220896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:14.220896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:14.220896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:14.220896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:14.220896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:14.220896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:14.220896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:14.220896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:14.220896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:09:14.220896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:09:14.220896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 609/1235] 10.222.0.78 () {56 vars in 1102 bytes} [Mon Sep 19 11:09:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:14.226485 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8d802-3ba2-4603-aa3b-5713a0f8f38a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:14.271638 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:14.323338 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8d802-3ba2-4603-aa3b-5713a0f8f38a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:14.356766 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace65d9-d64d-446d-a61f-6af8221ed2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:14.357384 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace65d9-d64d-446d-a61f-6af8221ed2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:14.357663 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace65d9-d64d-446d-a61f-6af8221ed2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:14.357932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace65d9-d64d-446d-a61f-6af8221ed2b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:14.358595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace65d9-d64d-446d-a61f-6af8221ed2b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:14.358903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace65d9-d64d-446d-a61f-6af8221ed2b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:14.360356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace65d9-d64d-446d-a61f-6af8221ed2b9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:14.368905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 627/1236] 10.222.0.78 () {56 vars in 1103 bytes} [Mon Sep 19 11:09:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:14.375346 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09c025-4053-4c9b-af80-4a3b7be2425e None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:14.460704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:14.476965 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09c025-4053-4c9b-af80-4a3b7be2425e None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:14.510715 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bafbf1-7eab-409a-8684-e6c141cde5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:14.511409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bafbf1-7eab-409a-8684-e6c141cde5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:14.511695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bafbf1-7eab-409a-8684-e6c141cde5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:14.511971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bafbf1-7eab-409a-8684-e6c141cde5d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:14.522585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bafbf1-7eab-409a-8684-e6c141cde5d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:14.524063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bafbf1-7eab-409a-8684-e6c141cde5d8 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:14.539628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-98bafbf1-7eab-409a-8684-e6c141cde5d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:14.539628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:14.539628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:14.539628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:14.539628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:14.539628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:14.539628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:14.539628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:14.539628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:14.539628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:14.539628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:14.539628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:14.539628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:09:14.539628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:09:14.539628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:09:14.539628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:09:14.540666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:14.540666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:14.540666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:14.540666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:14.540666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:14.540666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:14.540666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:14.540666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:14.540666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:14.540666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:14.540666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:14.540666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:14.540666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:14.540666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:14.540666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:14.540666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:14.540666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:09:14.541735 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:09:14.541735 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:14.541735 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:09:14.541735 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 610/1237] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:14.546780 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace65d9-d64d-446d-a61f-6af8221ed2b9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:14.645066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace65d9-d64d-446d-a61f-6af8221ed2b9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:14.654243 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:14.678434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b87fb3-25fc-4d73-942c-cd2290a5ce49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:14.679060 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b87fb3-25fc-4d73-942c-cd2290a5ce49 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:14.679370 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b87fb3-25fc-4d73-942c-cd2290a5ce49 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:14.679649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b87fb3-25fc-4d73-942c-cd2290a5ce49 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:14.680288 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b87fb3-25fc-4d73-942c-cd2290a5ce49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:14.680591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b87fb3-25fc-4d73-942c-cd2290a5ce49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:14.681955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b87fb3-25fc-4d73-942c-cd2290a5ce49 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:14.693236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 628/1238] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:14.699101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bafbf1-7eab-409a-8684-e6c141cde5d8 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:14.799630 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bafbf1-7eab-409a-8684-e6c141cde5d8 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:14.832937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2e10f-485c-4d4b-99f4-0975a93db52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:14.833651 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2e10f-485c-4d4b-99f4-0975a93db52f None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:14.833901 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2e10f-485c-4d4b-99f4-0975a93db52f None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:14.834178 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2e10f-485c-4d4b-99f4-0975a93db52f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:14.837461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:14.842675 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd2e10f-485c-4d4b-99f4-0975a93db52f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:14.844153 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd2e10f-485c-4d4b-99f4-0975a93db52f None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:14.855935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-fcd2e10f-485c-4d4b-99f4-0975a93db52f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:09:14.855935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:14.855935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:14.855935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:14.855935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:14.855935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:14.855935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:14.855935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:14.855935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:14.855935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:14.855935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:14.855935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:14.855935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:09:14.855935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:09:14.855935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:09:14.855935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:14.857115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:14.858281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:14.858281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:14.858281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:09:14.858281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:09:14.858281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:09:14.858281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:09:14.858281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:09:14.858281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:09:14.858281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:09:14.858281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:09:14.858281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 611/1239] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:14.862989 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b87fb3-25fc-4d73-942c-cd2290a5ce49 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:14.962698 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b87fb3-25fc-4d73-942c-cd2290a5ce49 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:14.996370 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932eac1-2afb-44c7-9c4a-4983c9ff69b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:14.996980 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932eac1-2afb-44c7-9c4a-4983c9ff69b1 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:14.997263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932eac1-2afb-44c7-9c4a-4983c9ff69b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:14.997539 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932eac1-2afb-44c7-9c4a-4983c9ff69b1 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:14.998268 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932eac1-2afb-44c7-9c4a-4983c9ff69b1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:14.998618 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932eac1-2afb-44c7-9c4a-4983c9ff69b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:15.000072 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932eac1-2afb-44c7-9c4a-4983c9ff69b1 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:15.017447 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 629/1240] 10.222.0.78 () {56 vars in 1143 bytes} [Mon Sep 19 11:09:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:15.024063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2e10f-485c-4d4b-99f4-0975a93db52f None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:15.125699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2e10f-485c-4d4b-99f4-0975a93db52f None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:15.159925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04eec8-bfce-498b-9a9b-70de899bdb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:15.160530 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04eec8-bfce-498b-9a9b-70de899bdb57 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:15.160807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04eec8-bfce-498b-9a9b-70de899bdb57 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:15.161079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04eec8-bfce-498b-9a9b-70de899bdb57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:15.221087 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:15.227333 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af04eec8-bfce-498b-9a9b-70de899bdb57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:15.228822 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af04eec8-bfce-498b-9a9b-70de899bdb57 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:15.327663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 612/1241] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:09:15.333830 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932eac1-2afb-44c7-9c4a-4983c9ff69b1 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:15.361001 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:15.441059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932eac1-2afb-44c7-9c4a-4983c9ff69b1 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:15.474122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a80e83-2750-4461-b50d-ff2e9147be4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:15.474680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a80e83-2750-4461-b50d-ff2e9147be4d None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:15.474959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a80e83-2750-4461-b50d-ff2e9147be4d None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:15.475263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a80e83-2750-4461-b50d-ff2e9147be4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:15.485074 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a80e83-2750-4461-b50d-ff2e9147be4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:15.486499 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a80e83-2750-4461-b50d-ff2e9147be4d None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:15.501519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-a9a80e83-2750-4461-b50d-ff2e9147be4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:15.501519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:15.501519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:15.501519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:15.501519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:15.501519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:15.501519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:15.501519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:15.501519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:15.501519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:15.501519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:15.501519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:15.501519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:09:15.501519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:09:15.501519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:09:15.501519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:09:15.502775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:15.502775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:15.502775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:15.502775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:15.502775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:15.502775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:15.502775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:15.502775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:15.502775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:15.502775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:15.502775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:15.502775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:15.502775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:15.502775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:15.502775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:15.502775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:15.502775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:09:15.505411 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:09:15.505411 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:15.505411 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:15.507420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 630/1242] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:15.508946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04eec8-bfce-498b-9a9b-70de899bdb57 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:15.600650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:15.606370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04eec8-bfce-498b-9a9b-70de899bdb57 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:15.640136 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa97ef-a578-4a13-9d11-4667df7e2fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:15.640742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa97ef-a578-4a13-9d11-4667df7e2fba None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:15.641021 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa97ef-a578-4a13-9d11-4667df7e2fba None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:15.641291 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa97ef-a578-4a13-9d11-4667df7e2fba None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:15.641928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa97ef-a578-4a13-9d11-4667df7e2fba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:15.642227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa97ef-a578-4a13-9d11-4667df7e2fba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:15.643620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa97ef-a578-4a13-9d11-4667df7e2fba None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:15.654515 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 613/1243] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:15.660441 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a80e83-2750-4461-b50d-ff2e9147be4d None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:15.739588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:15.760977 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a80e83-2750-4461-b50d-ff2e9147be4d None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:15.793706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5eac33-bfc7-4e27-b3f1-0ffd959b149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:15.794330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5eac33-bfc7-4e27-b3f1-0ffd959b149b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:15.794608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5eac33-bfc7-4e27-b3f1-0ffd959b149b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:15.794879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5eac33-bfc7-4e27-b3f1-0ffd959b149b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:15.801224 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5eac33-bfc7-4e27-b3f1-0ffd959b149b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:15.802647 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5eac33-bfc7-4e27-b3f1-0ffd959b149b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:15.814430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-ad5eac33-bfc7-4e27-b3f1-0ffd959b149b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:09:15.814430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:15.814430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:15.814430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:15.814430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:15.814430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:15.814430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:15.814430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:15.814430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:15.814430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:15.814430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:15.814430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:15.814430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:09:15.814430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:09:15.814430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:09:15.815588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:09:15.815588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:15.815588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:15.815588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:15.815588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:15.815588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:15.815588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:15.815588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:15.815588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:15.815588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:15.815588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:15.815588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:15.815588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:15.815588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:15.815588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:15.815588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:15.815588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:15.816531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:09:15.816531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:09:15.816531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:09:15.816531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:09:15.816531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:09:15.816531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:09:15.816531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:09:15.816531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:15.816531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 631/1244] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:09:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:15.821600 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa97ef-a578-4a13-9d11-4667df7e2fba None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:15.921250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa97ef-a578-4a13-9d11-4667df7e2fba None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:15.954079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f36c5-efee-4ba8-95eb-0f51364e9fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:15.954686 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f36c5-efee-4ba8-95eb-0f51364e9fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:15.954974 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f36c5-efee-4ba8-95eb-0f51364e9fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:15.955282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f36c5-efee-4ba8-95eb-0f51364e9fc8 None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:15.955977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f36c5-efee-4ba8-95eb-0f51364e9fc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:15.956304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f36c5-efee-4ba8-95eb-0f51364e9fc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:15.957722 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f36c5-efee-4ba8-95eb-0f51364e9fc8 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:15.970567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 614/1245] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:09:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:15.976968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5eac33-bfc7-4e27-b3f1-0ffd959b149b None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:16.076251 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5eac33-bfc7-4e27-b3f1-0ffd959b149b None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:16.109408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063f431-9a9a-4fd6-93ab-c6ac56690eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:16.110025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063f431-9a9a-4fd6-93ab-c6ac56690eec None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:16.110307 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063f431-9a9a-4fd6-93ab-c6ac56690eec None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:16.110579 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063f431-9a9a-4fd6-93ab-c6ac56690eec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:16.111337 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063f431-9a9a-4fd6-93ab-c6ac56690eec None admin] RBAC: Adding query filter params (role.id=28365abffd3d4ff6874bce2e0c561f1d, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=0427cdd3905a422cab4fed0aac187ce2) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:16.111709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063f431-9a9a-4fd6-93ab-c6ac56690eec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:16.113076 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063f431-9a9a-4fd6-93ab-c6ac56690eec None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:16.120012 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 632/1246] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:15 2022] GET /identity/v3/role_assignments?user.id=0427cdd3905a422cab4fed0aac187ce2&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=28365abffd3d4ff6874bce2e0c561f1d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:18.394519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dee3d-3c1f-4fa8-a42c-fd76211a5f42 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:18.395059 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dee3d-3c1f-4fa8-a42c-fd76211a5f42 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:18.395059 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dee3d-3c1f-4fa8-a42c-fd76211a5f42 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:18.396340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 615/1247] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:18.401913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1e8ff-f33d-4280-853d-f21df7e76c3b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:18.402347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1e8ff-f33d-4280-853d-f21df7e76c3b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:18.402518 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1e8ff-f33d-4280-853d-f21df7e76c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:18.409122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:18.422293 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-3ca1e8ff-f33d-4280-853d-f21df7e76c3b None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:18.426041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3ca1e8ff-f33d-4280-853d-f21df7e76c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:18.490004 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 633/1248] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:09:18.764965 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90dee3d-3c1f-4fa8-a42c-fd76211a5f42 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:18.783365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:18.869479 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90dee3d-3c1f-4fa8-a42c-fd76211a5f42 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:18.906111 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67694b9-fa1c-4ad1-a1f9-f71e18e6a894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:18.906859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67694b9-fa1c-4ad1-a1f9-f71e18e6a894 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:18.907233 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67694b9-fa1c-4ad1-a1f9-f71e18e6a894 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:18.907513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67694b9-fa1c-4ad1-a1f9-f71e18e6a894 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:18.918647 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67694b9-fa1c-4ad1-a1f9-f71e18e6a894 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:18.923629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-b67694b9-fa1c-4ad1-a1f9-f71e18e6a894 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:18.923629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:18.923629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:18.923629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:18.923629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:18.923629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:18.923629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:18.923629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:18.923629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:18.923629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:18.923629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:18.923629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:18.923629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:18.923629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:18.923629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:18.923629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:18.924740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:18.924740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:18.924740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:18.924740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:18.924740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:18.924740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:18.924740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:18.924740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:18.924740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:18.924740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:18.924740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:18.924740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:18.924740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:18.924740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:18.924740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:18.924740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:18.925931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:18.925931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:18.925931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:18.925931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:18.925931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:18.925931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:18.925931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:18.925931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:18.925931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:18.925931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:18.925931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:18.925931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:09:18.925931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 616/1249] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:09:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:18.931420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1e8ff-f33d-4280-853d-f21df7e76c3b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:19.028182 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1e8ff-f33d-4280-853d-f21df7e76c3b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:19.058600 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:19.063348 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217c535-4c6d-4335-b9d2-21f23b2cdc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:19.063960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217c535-4c6d-4335-b9d2-21f23b2cdc7c None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:19.064242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217c535-4c6d-4335-b9d2-21f23b2cdc7c None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:19.064517 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217c535-4c6d-4335-b9d2-21f23b2cdc7c None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:19.065186 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217c535-4c6d-4335-b9d2-21f23b2cdc7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:19.065489 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217c535-4c6d-4335-b9d2-21f23b2cdc7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:19.066888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217c535-4c6d-4335-b9d2-21f23b2cdc7c None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:19.074896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 634/1250] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:19.080993 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67694b9-fa1c-4ad1-a1f9-f71e18e6a894 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:19.171008 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:19.194525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67694b9-fa1c-4ad1-a1f9-f71e18e6a894 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:19.230593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44eb02-3381-4f3f-8dff-f4c022c254f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:19.231262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44eb02-3381-4f3f-8dff-f4c022c254f2 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:19.231544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44eb02-3381-4f3f-8dff-f4c022c254f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:19.231867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44eb02-3381-4f3f-8dff-f4c022c254f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:19.242270 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44eb02-3381-4f3f-8dff-f4c022c254f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:19.244044 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44eb02-3381-4f3f-8dff-f4c022c254f2 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:19.259576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-2f44eb02-3381-4f3f-8dff-f4c022c254f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:19.259576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:19.259576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:19.259576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:19.259576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:19.259576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:19.259576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:19.259576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:19.259576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:19.259576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:19.259576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:19.259576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:19.259576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:09:19.259576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:09:19.259576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:09:19.259576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:09:19.260842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:19.260842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:19.260842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:19.260842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:19.260842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:19.260842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:19.260842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:19.260842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:19.260842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:19.260842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:19.260842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:19.260842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:19.260842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:19.260842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:19.260842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:19.260842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:19.260842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:09:19.261984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:09:19.261984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:19.261984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:09:19.261984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 617/1251] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:19.266583 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217c535-4c6d-4335-b9d2-21f23b2cdc7c None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:19.364631 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217c535-4c6d-4335-b9d2-21f23b2cdc7c None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:19.396876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa470532-63e8-40f0-b644-8b2f8341b50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:19.398004 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa470532-63e8-40f0-b644-8b2f8341b50a None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:19.398278 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa470532-63e8-40f0-b644-8b2f8341b50a None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:19.398546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa470532-63e8-40f0-b644-8b2f8341b50a None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:19.399195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa470532-63e8-40f0-b644-8b2f8341b50a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:19.399502 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa470532-63e8-40f0-b644-8b2f8341b50a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:19.400850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa470532-63e8-40f0-b644-8b2f8341b50a None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:19.412580 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 635/1252] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:19.419115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44eb02-3381-4f3f-8dff-f4c022c254f2 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:19.518725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44eb02-3381-4f3f-8dff-f4c022c254f2 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:19.545272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:19.552883 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6edd33-3009-4afd-a08a-6fe957f4835e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:19.553506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6edd33-3009-4afd-a08a-6fe957f4835e None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:19.553781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6edd33-3009-4afd-a08a-6fe957f4835e None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:19.554052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6edd33-3009-4afd-a08a-6fe957f4835e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:19.561388 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6edd33-3009-4afd-a08a-6fe957f4835e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:19.562877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6edd33-3009-4afd-a08a-6fe957f4835e None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:19.574349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-7f6edd33-3009-4afd-a08a-6fe957f4835e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:09:19.574349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:19.574349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:19.574349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:19.574349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:19.574349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:19.574349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:19.574349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:19.574349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:19.574349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:19.574349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:19.574349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:19.574349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:09:19.574349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:09:19.574349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:09:19.574349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:19.575588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:19.580956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:19.580956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:19.580956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:09:19.580956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:09:19.580956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:09:19.580956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:09:19.580956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:09:19.580956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:09:19.580956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:09:19.580956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:09:19.580956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 618/1253] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:19.581589 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa470532-63e8-40f0-b644-8b2f8341b50a None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:19.636779 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:19.679495 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa470532-63e8-40f0-b644-8b2f8341b50a None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:19.713148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569846ef-f113-48a4-9516-409decccb3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:19.713762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569846ef-f113-48a4-9516-409decccb3be None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:19.714047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569846ef-f113-48a4-9516-409decccb3be None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:19.714325 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569846ef-f113-48a4-9516-409decccb3be None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:19.715056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569846ef-f113-48a4-9516-409decccb3be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:19.715397 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569846ef-f113-48a4-9516-409decccb3be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:19.716835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569846ef-f113-48a4-9516-409decccb3be None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:19.733545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 636/1254] 10.222.0.78 () {56 vars in 1143 bytes} [Mon Sep 19 11:09:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:19.739439 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6edd33-3009-4afd-a08a-6fe957f4835e None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:19.838697 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6edd33-3009-4afd-a08a-6fe957f4835e None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:19.872988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60632451-0e45-4db5-b4d7-1f50e210aed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:19.873601 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60632451-0e45-4db5-b4d7-1f50e210aed0 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:19.873882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60632451-0e45-4db5-b4d7-1f50e210aed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:19.874156 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60632451-0e45-4db5-b4d7-1f50e210aed0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:19.928972 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:19.940788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60632451-0e45-4db5-b4d7-1f50e210aed0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:19.942241 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60632451-0e45-4db5-b4d7-1f50e210aed0 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:20.039767 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 619/1255] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:09:20.045998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569846ef-f113-48a4-9516-409decccb3be None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:20.145246 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569846ef-f113-48a4-9516-409decccb3be None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:20.157953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:20.183459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594db6b-143e-4da2-8792-804496a81ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:20.184045 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594db6b-143e-4da2-8792-804496a81ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:20.184931 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594db6b-143e-4da2-8792-804496a81ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:20.185216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594db6b-143e-4da2-8792-804496a81ab0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:20.197381 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594db6b-143e-4da2-8792-804496a81ab0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:20.198884 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594db6b-143e-4da2-8792-804496a81ab0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:20.213130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-c594db6b-143e-4da2-8792-804496a81ab0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:20.213130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:20.213130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:20.213130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:20.213130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:20.213130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:20.213130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:20.213130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:20.213130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:20.213130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:20.213130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:20.213130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:20.213130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:09:20.213130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:09:20.213130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:09:20.213130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:09:20.214248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:20.214248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:20.214248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:20.214248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:20.214248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:20.214248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:20.214248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:20.214248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:20.214248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:20.214248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:20.214248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:20.214248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:20.214248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:20.214248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:20.214248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:20.214248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:20.214248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:09:20.215347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:09:20.215347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:20.215347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:20.215347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 637/1256] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:20.220222 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60632451-0e45-4db5-b4d7-1f50e210aed0 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:20.308780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:20.320106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60632451-0e45-4db5-b4d7-1f50e210aed0 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:20.354028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0f9a1-3560-4d14-93ca-54a57c3c7877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:20.354710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0f9a1-3560-4d14-93ca-54a57c3c7877 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:20.355009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0f9a1-3560-4d14-93ca-54a57c3c7877 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:20.355290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0f9a1-3560-4d14-93ca-54a57c3c7877 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:20.355928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0f9a1-3560-4d14-93ca-54a57c3c7877 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:20.356230 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0f9a1-3560-4d14-93ca-54a57c3c7877 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:20.362979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0f9a1-3560-4d14-93ca-54a57c3c7877 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:20.369768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 620/1257] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:20.375571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594db6b-143e-4da2-8792-804496a81ab0 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:20.476694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594db6b-143e-4da2-8792-804496a81ab0 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:20.511232 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908a928-190d-41f2-acde-0d916d9e3cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:20.511902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908a928-190d-41f2-acde-0d916d9e3cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:20.512189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908a928-190d-41f2-acde-0d916d9e3cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:20.512466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908a928-190d-41f2-acde-0d916d9e3cd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:20.519187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908a928-190d-41f2-acde-0d916d9e3cd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:20.520671 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908a928-190d-41f2-acde-0d916d9e3cd2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:20.531778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-c908a928-190d-41f2-acde-0d916d9e3cd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:09:20.531778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:20.531778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:20.531778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:20.531778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:20.531778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:20.531778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:20.531778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:20.531778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:20.531778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:20.531778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:20.531778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:20.531778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:09:20.531778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:09:20.531778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:09:20.532863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:09:20.532863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:20.532863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:20.532863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:20.532863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:20.532863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:20.532863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:20.532863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:20.532863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:20.532863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:20.532863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:20.532863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:20.532863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:20.532863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:20.532863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:20.532863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:20.532863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:20.533974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:09:20.533974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:09:20.533974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:09:20.533974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:09:20.533974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:09:20.533974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:09:20.533974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:09:20.533974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:20.533974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 638/1258] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:09:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:20.539410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0f9a1-3560-4d14-93ca-54a57c3c7877 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:20.640124 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0f9a1-3560-4d14-93ca-54a57c3c7877 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:20.675694 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150aaec-5323-4c84-99d1-fc2f33e3cfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:20.676296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150aaec-5323-4c84-99d1-fc2f33e3cfff None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:20.676573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150aaec-5323-4c84-99d1-fc2f33e3cfff None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:20.676848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150aaec-5323-4c84-99d1-fc2f33e3cfff None admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:20.677539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150aaec-5323-4c84-99d1-fc2f33e3cfff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:20.677872 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150aaec-5323-4c84-99d1-fc2f33e3cfff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:20.679450 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150aaec-5323-4c84-99d1-fc2f33e3cfff None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:20.691593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 621/1259] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:09:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:20.697716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908a928-190d-41f2-acde-0d916d9e3cd2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:20.723698 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:20.797887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908a928-190d-41f2-acde-0d916d9e3cd2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:20.831538 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca9524-e651-48c9-9340-2791bd8139a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:20.832155 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca9524-e651-48c9-9340-2791bd8139a0 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:20.832434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca9524-e651-48c9-9340-2791bd8139a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:20.832705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca9524-e651-48c9-9340-2791bd8139a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:20.833428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca9524-e651-48c9-9340-2791bd8139a0 None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=0427cdd3905a422cab4fed0aac187ce2) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:20.833737 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca9524-e651-48c9-9340-2791bd8139a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:20.835745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca9524-e651-48c9-9340-2791bd8139a0 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:20.841237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 639/1260] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:20 2022] GET /identity/v3/role_assignments?user.id=0427cdd3905a422cab4fed0aac187ce2&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:22.950122 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77ddc2-d6af-46b4-b7e5-41fbdce9ad6d None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:22.950712 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77ddc2-d6af-46b4-b7e5-41fbdce9ad6d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:22.950712 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77ddc2-d6af-46b4-b7e5-41fbdce9ad6d None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:22.952010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 622/1261] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:09:22.957428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abc99d-42d4-474b-b3f1-acb8917249ff None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:22.957823 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abc99d-42d4-474b-b3f1-acb8917249ff None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:22.957990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abc99d-42d4-474b-b3f1-acb8917249ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:22.976244 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-57abc99d-42d4-474b-b3f1-acb8917249ff None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:22.977460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-57abc99d-42d4-474b-b3f1-acb8917249ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:23.042603 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 640/1262] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:09:23.311771 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77ddc2-d6af-46b4-b7e5-41fbdce9ad6d None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:23.322572 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:23.413460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77ddc2-d6af-46b4-b7e5-41fbdce9ad6d None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:23.448569 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663bf9a-57ad-4b8c-ac7b-c72504fd42ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:23.449192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663bf9a-57ad-4b8c-ac7b-c72504fd42ed None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:23.449470 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663bf9a-57ad-4b8c-ac7b-c72504fd42ed None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:23.449735 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663bf9a-57ad-4b8c-ac7b-c72504fd42ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:23.459636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663bf9a-57ad-4b8c-ac7b-c72504fd42ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:23.464431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-b663bf9a-57ad-4b8c-ac7b-c72504fd42ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:23.464431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:23.464431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:23.464431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:23.464431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:23.464431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:23.464431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:23.464431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:23.464431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:23.464431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:23.464431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:23.464431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:23.464431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:23.464431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:23.464431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:23.464431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:23.467880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:23.467880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:23.467880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:23.467880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:23.467880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:23.467880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:23.467880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:23.467880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:23.467880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:23.467880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:23.467880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:23.467880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:23.467880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:23.467880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:23.467880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:23.467880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:23.468995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:23.468995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:23.468995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:23.468995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:23.468995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:23.468995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:23.468995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:23.468995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:23.468995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:23.468995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:23.468995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:23.468995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:09:23.468995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 623/1263] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:09:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:23.471589 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57abc99d-42d4-474b-b3f1-acb8917249ff None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:23.489822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:23.567197 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57abc99d-42d4-474b-b3f1-acb8917249ff None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:23.601006 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101b13c-577d-48fc-afbd-a96342f4efe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:23.601643 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101b13c-577d-48fc-afbd-a96342f4efe5 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:23.601922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101b13c-577d-48fc-afbd-a96342f4efe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:23.602198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101b13c-577d-48fc-afbd-a96342f4efe5 None admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:23.602856 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101b13c-577d-48fc-afbd-a96342f4efe5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:23.603246 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101b13c-577d-48fc-afbd-a96342f4efe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:23.604628 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101b13c-577d-48fc-afbd-a96342f4efe5 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:23.612464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 641/1264] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:23.618377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663bf9a-57ad-4b8c-ac7b-c72504fd42ed None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:23.689302 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:23.722683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663bf9a-57ad-4b8c-ac7b-c72504fd42ed None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:23.755997 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976dd73-dece-47b5-8306-9b49f8531768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:23.757173 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976dd73-dece-47b5-8306-9b49f8531768 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:23.757449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976dd73-dece-47b5-8306-9b49f8531768 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:23.757718 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976dd73-dece-47b5-8306-9b49f8531768 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:23.768053 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976dd73-dece-47b5-8306-9b49f8531768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:23.781139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976dd73-dece-47b5-8306-9b49f8531768 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:23.783379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-d976dd73-dece-47b5-8306-9b49f8531768 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:23.783379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:23.783379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:23.783379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:23.783379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:23.783379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:23.783379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:23.783379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:23.783379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:23.783379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:23.783379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:23.783379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:23.783379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:09:23.783379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:09:23.783379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:09:23.783379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:09:23.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:23.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:23.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:23.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:23.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:23.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:23.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:23.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:23.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:23.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:23.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:23.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:23.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:23.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:23.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:23.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:23.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:09:23.785632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:09:23.785632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:23.785632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:09:23.785632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 624/1265] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:23.790362 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101b13c-577d-48fc-afbd-a96342f4efe5 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:23.871551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:23.890869 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101b13c-577d-48fc-afbd-a96342f4efe5 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:23.924887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c1a47-58c0-4dd7-9968-334cf69e91e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:23.925494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c1a47-58c0-4dd7-9968-334cf69e91e9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:23.925780 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c1a47-58c0-4dd7-9968-334cf69e91e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:23.926053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c1a47-58c0-4dd7-9968-334cf69e91e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:23.926690 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c1a47-58c0-4dd7-9968-334cf69e91e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:23.926992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c1a47-58c0-4dd7-9968-334cf69e91e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:23.928935 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c1a47-58c0-4dd7-9968-334cf69e91e9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:23.940092 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 642/1266] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:23.946412 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976dd73-dece-47b5-8306-9b49f8531768 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:24.045594 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976dd73-dece-47b5-8306-9b49f8531768 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:24.065423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:24.079085 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8c52e-00b2-4b58-a5cd-333ec4a3de06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:24.085182 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8c52e-00b2-4b58-a5cd-333ec4a3de06 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:24.085182 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8c52e-00b2-4b58-a5cd-333ec4a3de06 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:24.085182 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8c52e-00b2-4b58-a5cd-333ec4a3de06 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:24.088273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8c52e-00b2-4b58-a5cd-333ec4a3de06 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:24.089722 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8c52e-00b2-4b58-a5cd-333ec4a3de06 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:24.101428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-28c8c52e-00b2-4b58-a5cd-333ec4a3de06 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:09:24.101428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:24.101428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:24.101428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:24.101428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:24.101428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:24.101428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:24.101428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:24.101428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:24.101428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:24.101428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:24.101428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:24.101428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:09:24.101428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:09:24.101428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:09:24.101428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:24.102536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:24.103678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:24.103678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:24.103678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:09:24.103678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:09:24.103678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:09:24.103678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:09:24.103678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:09:24.103678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:09:24.103678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:09:24.103678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:09:24.103678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 625/1267] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:24.108629 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c1a47-58c0-4dd7-9968-334cf69e91e9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:24.209843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c1a47-58c0-4dd7-9968-334cf69e91e9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:24.243826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be7f8a1-d80b-4a83-b5b2-87a5ec6da0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:24.244435 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7f8a1-d80b-4a83-b5b2-87a5ec6da0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:24.244714 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7f8a1-d80b-4a83-b5b2-87a5ec6da0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:24.244985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7f8a1-d80b-4a83-b5b2-87a5ec6da0d7 None admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:24.245708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be7f8a1-d80b-4a83-b5b2-87a5ec6da0d7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:24.246066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be7f8a1-d80b-4a83-b5b2-87a5ec6da0d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:24.247531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be7f8a1-d80b-4a83-b5b2-87a5ec6da0d7 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:24.256610 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:24.264553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 643/1268] 10.222.0.78 () {56 vars in 1143 bytes} [Mon Sep 19 11:09:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:24.270578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8c52e-00b2-4b58-a5cd-333ec4a3de06 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:24.371595 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8c52e-00b2-4b58-a5cd-333ec4a3de06 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:24.405847 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2b209-69d8-493e-aa0a-0728825a9162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:24.406473 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2b209-69d8-493e-aa0a-0728825a9162 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:24.406748 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2b209-69d8-493e-aa0a-0728825a9162 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:24.407019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2b209-69d8-493e-aa0a-0728825a9162 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:24.417605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2b209-69d8-493e-aa0a-0728825a9162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:24.419043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2b209-69d8-493e-aa0a-0728825a9162 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:24.433808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-f3a2b209-69d8-493e-aa0a-0728825a9162 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:24.433808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:24.433808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:24.433808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:24.433808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:24.433808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:24.433808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:24.433808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:24.433808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:24.433808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:24.433808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:24.433808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:24.433808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:09:24.433808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:09:24.433808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:09:24.433808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:09:24.440942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:24.440942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:24.440942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:24.440942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:24.440942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:24.440942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:24.440942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:24.440942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:24.440942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:24.440942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:24.440942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:24.440942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:24.440942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:24.440942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:24.440942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:24.440942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:24.440942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:09:24.442045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:09:24.442045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:24.442045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:09:24.442045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 626/1269] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:24.442334 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be7f8a1-d80b-4a83-b5b2-87a5ec6da0d7 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:24.539311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be7f8a1-d80b-4a83-b5b2-87a5ec6da0d7 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:24.573510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfea6f-d7de-469f-b3b7-375117b0e887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:24.574115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfea6f-d7de-469f-b3b7-375117b0e887 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:24.574400 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfea6f-d7de-469f-b3b7-375117b0e887 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:24.574675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfea6f-d7de-469f-b3b7-375117b0e887 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:24.575349 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfea6f-d7de-469f-b3b7-375117b0e887 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:24.580933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfea6f-d7de-469f-b3b7-375117b0e887 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:24.580933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfea6f-d7de-469f-b3b7-375117b0e887 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:24.588852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 644/1270] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:24.594744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2b209-69d8-493e-aa0a-0728825a9162 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:24.626991 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:24.697976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2b209-69d8-493e-aa0a-0728825a9162 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:24.732751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72d20c-befb-49a0-98a5-1f268416cf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:24.733385 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72d20c-befb-49a0-98a5-1f268416cf34 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:24.733670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72d20c-befb-49a0-98a5-1f268416cf34 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:24.733946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72d20c-befb-49a0-98a5-1f268416cf34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:24.741292 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72d20c-befb-49a0-98a5-1f268416cf34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:24.742777 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72d20c-befb-49a0-98a5-1f268416cf34 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:24.754289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-7b72d20c-befb-49a0-98a5-1f268416cf34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:09:24.754289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:24.754289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:24.754289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:24.754289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:24.754289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:24.754289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:24.754289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:24.754289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:24.754289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:24.754289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:24.754289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:24.754289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:09:24.754289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:09:24.754289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:09:24.756988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:09:24.756988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:24.756988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:24.756988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:24.756988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:24.756988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:24.756988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:24.756988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:24.756988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:24.756988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:24.756988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:24.756988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:24.756988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:24.756988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:24.756988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:24.756988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:24.756988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:24.758100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:09:24.758100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:09:24.758100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:09:24.758100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:09:24.758100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:09:24.758100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:09:24.758100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:09:24.758100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:09:24.758100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 627/1271] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:09:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:24.761466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfea6f-d7de-469f-b3b7-375117b0e887 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:24.821931 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:24.862885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfea6f-d7de-469f-b3b7-375117b0e887 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:24.896011 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14f385-53ab-44a6-bc46-3682189241b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:24.896746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14f385-53ab-44a6-bc46-3682189241b2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:24.897093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14f385-53ab-44a6-bc46-3682189241b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:24.897412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14f385-53ab-44a6-bc46-3682189241b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:24.898143 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14f385-53ab-44a6-bc46-3682189241b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:24.898522 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14f385-53ab-44a6-bc46-3682189241b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:24.900089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14f385-53ab-44a6-bc46-3682189241b2 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:24.913080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 645/1272] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:09:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:24.919455 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72d20c-befb-49a0-98a5-1f268416cf34 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:25.011865 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:25.020626 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72d20c-befb-49a0-98a5-1f268416cf34 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:25.060388 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537fed38-f22b-4f20-b1dd-5907a911f533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:25.061055 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fed38-f22b-4f20-b1dd-5907a911f533 None admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:25.061296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fed38-f22b-4f20-b1dd-5907a911f533 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:25.061560 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fed38-f22b-4f20-b1dd-5907a911f533 None admin] PATH_INFO: `/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/0427cdd3905a422cab4fed0aac187ce2/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:25.063520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537fed38-f22b-4f20-b1dd-5907a911f533 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8563e86b7c94ff29e39fb20a7969551, user_id=0427cdd3905a422cab4fed0aac187ce2, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:25.064943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537fed38-f22b-4f20-b1dd-5907a911f533 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:25.142425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 628/1273] 10.222.0.78 () {58 vars in 1439 bytes} [Mon Sep 19 11:09:24 2022] PUT /identity/v3/projects/a8563e86b7c94ff29e39fb20a7969551/users/0427cdd3905a422cab4fed0aac187ce2/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:09:27.303956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9918a-8665-496a-97e5-cff2cde9331e None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:27.304516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9918a-8665-496a-97e5-cff2cde9331e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:27.304859 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9918a-8665-496a-97e5-cff2cde9331e None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:27.307436 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 646/1274] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:09:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:27.314113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6f24f-a705-49a0-85a9-5183bbedc837 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:27.314113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6f24f-a705-49a0-85a9-5183bbedc837 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:27.314113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6f24f-a705-49a0-85a9-5183bbedc837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:27.332063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-35f6f24f-a705-49a0-85a9-5183bbedc837 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:27.333261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-35f6f24f-a705-49a0-85a9-5183bbedc837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:27.401535 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 629/1275] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:09:27.680660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9918a-8665-496a-97e5-cff2cde9331e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:27.783310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9918a-8665-496a-97e5-cff2cde9331e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:27.800057 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:27.816205 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487e105-ba97-47bc-a164-7e15822cd7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:27.816835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487e105-ba97-47bc-a164-7e15822cd7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:27.817297 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487e105-ba97-47bc-a164-7e15822cd7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:27.817579 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487e105-ba97-47bc-a164-7e15822cd7d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:27.828034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487e105-ba97-47bc-a164-7e15822cd7d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:27.833037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-7487e105-ba97-47bc-a164-7e15822cd7d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:27.833037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:27.833037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:27.833037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:27.833037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:27.833037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:27.833037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:27.833037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:27.833037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:27.833037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:27.833037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:27.833037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:27.833037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:27.833037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:27.833037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:27.833037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:27.834782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:27.834782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:27.834782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:27.834782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:27.834782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:27.834782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:27.834782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:27.834782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:27.834782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:27.834782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:27.834782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:27.834782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:27.834782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:27.834782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:27.834782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:27.834782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:27.835901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:27.835901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:27.835901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:27.835901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:27.835901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:27.835901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:27.835901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:27.835901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:27.835901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:27.835901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:27.835901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:27.835901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:27.835901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 647/1276] 10.222.0.78 () {56 vars in 1096 bytes} [Mon Sep 19 11:09:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:27.840810 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6f24f-a705-49a0-85a9-5183bbedc837 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:27.921624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:27.939322 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6f24f-a705-49a0-85a9-5183bbedc837 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:27.983055 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e94c2-471d-48ef-85e5-89e2070b64df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:27.983687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e94c2-471d-48ef-85e5-89e2070b64df None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:27.983961 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e94c2-471d-48ef-85e5-89e2070b64df None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:27.984229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e94c2-471d-48ef-85e5-89e2070b64df None admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:27.984885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e94c2-471d-48ef-85e5-89e2070b64df None admin] RBAC: Adding query filter params (name=admin) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:27.985181 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e94c2-471d-48ef-85e5-89e2070b64df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:27.986531 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e94c2-471d-48ef-85e5-89e2070b64df None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:27.994582 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 630/1277] 10.222.0.78 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:28.000614 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487e105-ba97-47bc-a164-7e15822cd7d2 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:28.100145 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487e105-ba97-47bc-a164-7e15822cd7d2 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:28.135507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375a981-44fd-4ec9-bc03-8cf1dbf042a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:28.136163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375a981-44fd-4ec9-bc03-8cf1dbf042a9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:28.136447 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375a981-44fd-4ec9-bc03-8cf1dbf042a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:28.136723 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375a981-44fd-4ec9-bc03-8cf1dbf042a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:28.146735 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375a981-44fd-4ec9-bc03-8cf1dbf042a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:28.148258 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375a981-44fd-4ec9-bc03-8cf1dbf042a9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:28.162578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-1375a981-44fd-4ec9-bc03-8cf1dbf042a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:28.162578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:28.162578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:28.162578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:28.162578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:28.162578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:28.162578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:28.162578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:28.162578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:28.162578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:28.162578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:28.162578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:28.162578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:09:28.162578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:09:28.162578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:09:28.162578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:09:28.163746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:28.163746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:28.163746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:28.163746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:28.163746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:28.163746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:28.163746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:28.163746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:28.163746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:28.163746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:28.163746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:28.163746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:28.163746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:28.163746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:28.163746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:28.163746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:28.163746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:09:28.164712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:09:28.164712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:28.164712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:28.164712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 648/1278] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:28.169502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e94c2-471d-48ef-85e5-89e2070b64df None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:28.267552 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e94c2-471d-48ef-85e5-89e2070b64df None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:28.299771 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d8eed-d2d4-4c4a-ac8d-047abc5d6fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:28.300376 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d8eed-d2d4-4c4a-ac8d-047abc5d6fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:28.300652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d8eed-d2d4-4c4a-ac8d-047abc5d6fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:28.300924 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d8eed-d2d4-4c4a-ac8d-047abc5d6fb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:28.301555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d8eed-d2d4-4c4a-ac8d-047abc5d6fb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:28.301852 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d8eed-d2d4-4c4a-ac8d-047abc5d6fb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:28.303766 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d8eed-d2d4-4c4a-ac8d-047abc5d6fb8 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:28.315667 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 631/1279] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:28.321777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375a981-44fd-4ec9-bc03-8cf1dbf042a9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:28.352473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:28.420082 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375a981-44fd-4ec9-bc03-8cf1dbf042a9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:28.453987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4d74e-c74b-48fc-be5d-f5b967cc9e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:28.454608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4d74e-c74b-48fc-be5d-f5b967cc9e42 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:28.454881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4d74e-c74b-48fc-be5d-f5b967cc9e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:28.455148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4d74e-c74b-48fc-be5d-f5b967cc9e42 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:28.461912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4d74e-c74b-48fc-be5d-f5b967cc9e42 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:28.463358 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4d74e-c74b-48fc-be5d-f5b967cc9e42 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:28.474463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-d4e4d74e-c74b-48fc-be5d-f5b967cc9e42 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:09:28.474463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:28.474463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:28.474463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:28.474463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:28.474463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:28.474463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:28.474463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:28.474463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:28.474463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:28.474463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:28.474463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:28.474463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:09:28.474463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:09:28.474463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:09:28.474463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:28.477747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:28.479134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:28.479134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:28.479134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:09:28.479134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:09:28.479134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:09:28.479134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:09:28.479134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:09:28.479134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:09:28.479134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:09:28.479134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:28.479134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 649/1280] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:28.482039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d8eed-d2d4-4c4a-ac8d-047abc5d6fb8 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:28.485930 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:28.581027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d8eed-d2d4-4c4a-ac8d-047abc5d6fb8 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:28.616395 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd241e-151d-470f-a32f-11458e740530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:28.617001 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd241e-151d-470f-a32f-11458e740530 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:28.617276 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd241e-151d-470f-a32f-11458e740530 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:28.617545 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd241e-151d-470f-a32f-11458e740530 None admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:28.618272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd241e-151d-470f-a32f-11458e740530 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:28.618828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd241e-151d-470f-a32f-11458e740530 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:28.620283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd241e-151d-470f-a32f-11458e740530 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:28.637512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 632/1281] 10.222.0.78 () {56 vars in 1143 bytes} [Mon Sep 19 11:09:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:28.643377 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4d74e-c74b-48fc-be5d-f5b967cc9e42 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:28.731685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:28.741654 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4d74e-c74b-48fc-be5d-f5b967cc9e42 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:28.774797 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0afc78-7c97-40ad-a7d6-008cc042d5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:28.775433 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0afc78-7c97-40ad-a7d6-008cc042d5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:28.775718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0afc78-7c97-40ad-a7d6-008cc042d5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:28.775989 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0afc78-7c97-40ad-a7d6-008cc042d5a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:28.841716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0afc78-7c97-40ad-a7d6-008cc042d5a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:28.843137 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0afc78-7c97-40ad-a7d6-008cc042d5a9 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:28.929552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:28.939434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 650/1282] 10.222.0.78 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:09:28.945412 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd241e-151d-470f-a32f-11458e740530 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:29.008930 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:29.058620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd241e-151d-470f-a32f-11458e740530 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:29.092503 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223c38c-f738-47d3-b4ac-35f08effa68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:29.093061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223c38c-f738-47d3-b4ac-35f08effa68a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:29.093340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223c38c-f738-47d3-b4ac-35f08effa68a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:29.093609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223c38c-f738-47d3-b4ac-35f08effa68a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:29.104631 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223c38c-f738-47d3-b4ac-35f08effa68a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:29.106069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223c38c-f738-47d3-b4ac-35f08effa68a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:29.126788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-f223c38c-f738-47d3-b4ac-35f08effa68a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:29.126788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:29.126788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:29.126788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:29.126788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:29.126788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:29.126788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:29.126788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:29.126788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:29.126788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:29.126788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:29.126788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:29.126788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:09:29.126788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:09:29.126788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:09:29.126788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:09:29.127856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:29.127856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:29.127856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:29.127856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:29.127856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:29.127856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:29.127856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:29.127856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:29.127856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:29.127856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:29.127856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:29.127856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:29.127856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:29.127856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:29.127856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:29.127856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:29.127856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:09:29.128834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:09:29.128834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:29.128834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:09:29.128834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 633/1283] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:29.134124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0afc78-7c97-40ad-a7d6-008cc042d5a9 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:29.243919 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0afc78-7c97-40ad-a7d6-008cc042d5a9 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:29.277637 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d725a0-0075-4a58-9358-73a76f16a6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:29.278242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d725a0-0075-4a58-9358-73a76f16a6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:29.278520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d725a0-0075-4a58-9358-73a76f16a6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:29.278788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d725a0-0075-4a58-9358-73a76f16a6e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:29.279456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d725a0-0075-4a58-9358-73a76f16a6e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:29.279756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d725a0-0075-4a58-9358-73a76f16a6e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:29.281119 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d725a0-0075-4a58-9358-73a76f16a6e8 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:29.293168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 651/1284] 10.222.0.78 () {56 vars in 1109 bytes} [Mon Sep 19 11:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:29.298918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223c38c-f738-47d3-b4ac-35f08effa68a None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:29.404171 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223c38c-f738-47d3-b4ac-35f08effa68a None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:29.418129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:29.438709 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c237ba-dfb7-457d-9c06-81626391ef2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:29.439367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c237ba-dfb7-457d-9c06-81626391ef2a None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:29.439642 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c237ba-dfb7-457d-9c06-81626391ef2a None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:29.439910 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c237ba-dfb7-457d-9c06-81626391ef2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:29.447400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c237ba-dfb7-457d-9c06-81626391ef2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:29.448854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c237ba-dfb7-457d-9c06-81626391ef2a None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:29.461114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application [None req-26c237ba-dfb7-457d-9c06-81626391ef2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:09:29.461114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:29.461114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:29.461114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:29.461114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:29.461114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:29.461114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:29.461114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:29.461114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:29.461114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:29.461114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:29.461114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:29.461114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:09:29.461114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:09:29.461114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:09:29.462113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:09:29.462113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:29.462113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:29.462113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:29.462113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:29.462113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:29.462113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:29.462113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:29.462113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:29.462113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:29.462113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:29.462113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:29.462113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:29.462113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:29.462113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:29.462113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:29.462113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:29.463465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:09:29.463465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:09:29.463465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:09:29.463465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:09:29.463465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:09:29.463465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:09:29.463465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:09:29.463465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: ERROR keystone.server.flask.application Sep 19 11:09:29.463465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 634/1285] 10.222.0.78 () {56 vars in 1111 bytes} [Mon Sep 19 11:09:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:29.468193 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d725a0-0075-4a58-9358-73a76f16a6e8 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:29.519493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:29.569541 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d725a0-0075-4a58-9358-73a76f16a6e8 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:29.602348 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d13c4-b9a6-483d-924f-0fcf01370f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:29.602902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d13c4-b9a6-483d-924f-0fcf01370f9b None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:29.603206 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d13c4-b9a6-483d-924f-0fcf01370f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:29.603489 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d13c4-b9a6-483d-924f-0fcf01370f9b None admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:29.604175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d13c4-b9a6-483d-924f-0fcf01370f9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:29.604708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d13c4-b9a6-483d-924f-0fcf01370f9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:29.606155 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d13c4-b9a6-483d-924f-0fcf01370f9b None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:29.620224 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 652/1286] 10.222.0.78 () {56 vars in 1148 bytes} [Mon Sep 19 11:09:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:29.626675 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c237ba-dfb7-457d-9c06-81626391ef2a None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:29.738493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c237ba-dfb7-457d-9c06-81626391ef2a None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:29.776498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf38692-f470-476c-b01a-d794202f3e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:29.777111 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf38692-f470-476c-b01a-d794202f3e1c None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:29.777391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf38692-f470-476c-b01a-d794202f3e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:29.777664 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf38692-f470-476c-b01a-d794202f3e1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:29.778399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf38692-f470-476c-b01a-d794202f3e1c None admin] RBAC: Adding query filter params (role.id=ee5bfb3e2bf64cf0847ac93b014672d1, scope.project.id=a8563e86b7c94ff29e39fb20a7969551, user.id=0427cdd3905a422cab4fed0aac187ce2) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:29.778709 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf38692-f470-476c-b01a-d794202f3e1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:29.780221 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf38692-f470-476c-b01a-d794202f3e1c None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:29.786778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 635/1287] 10.222.0.78 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:29 2022] GET /identity/v3/role_assignments?user.id=0427cdd3905a422cab4fed0aac187ce2&scope.project.id=a8563e86b7c94ff29e39fb20a7969551&role.id=ee5bfb3e2bf64cf0847ac93b014672d1 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:32.008342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f0ead-0e28-47d8-ae9b-644d2993dd5b None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:32.008892 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f0ead-0e28-47d8-ae9b-644d2993dd5b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:32.008892 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f0ead-0e28-47d8-ae9b-644d2993dd5b None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:32.010407 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 653/1288] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:32.015824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ad79d-45f1-4e82-98a0-3f08f82b679e None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:32.017657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ad79d-45f1-4e82-98a0-3f08f82b679e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:32.017657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ad79d-45f1-4e82-98a0-3f08f82b679e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:32.035906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-fe9ad79d-45f1-4e82-98a0-3f08f82b679e None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:32.037740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-fe9ad79d-45f1-4e82-98a0-3f08f82b679e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:32.068063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:32.104055 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 636/1289] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:09:32.370223 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f0ead-0e28-47d8-ae9b-644d2993dd5b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:32.471724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f0ead-0e28-47d8-ae9b-644d2993dd5b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:32.483279 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:32.507142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c95d4f-d886-497a-90b6-1f18bb381322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:32.507795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c95d4f-d886-497a-90b6-1f18bb381322 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:32.508079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c95d4f-d886-497a-90b6-1f18bb381322 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:32.508354 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c95d4f-d886-497a-90b6-1f18bb381322 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:32.508939 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c95d4f-d886-497a-90b6-1f18bb381322 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:32.510295 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c95d4f-d886-497a-90b6-1f18bb381322 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:32.519416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-23c95d4f-d886-497a-90b6-1f18bb381322 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:09:32.519416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:32.519416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:09:32.519416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:09:32.519416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:09:32.519416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:09:32.519416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:09:32.519416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:09:32.519416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:09:32.519416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:32.519416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:09:32.519416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:32.519416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:32.519416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:32.519416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:32.519416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:32.519416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:32.520667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:32.520667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:32.520667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:32.520667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:32.520667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:32.520667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:32.520667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:09:32.520667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:09:32.520667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:09:32.520667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:09:32.520667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:32.520667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:32.520667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:32.520667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:32.520667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:32.520667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:32.521854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:32.521854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:32.521854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:32.521854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:32.521854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:32.521854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:32.521854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:32.521854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:32.521854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:32.521854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:32.521854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:09:32.521854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:09:32.521854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:09:32.521854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:32.521854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 654/1290] 10.222.0.78 () {56 vars in 1117 bytes} [Mon Sep 19 11:09:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:32.526575 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ad79d-45f1-4e82-98a0-3f08f82b679e None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:32.628298 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ad79d-45f1-4e82-98a0-3f08f82b679e None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:32.661344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37597a9-acec-488a-9ce0-968d11e43a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:32.661952 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37597a9-acec-488a-9ce0-968d11e43a12 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:32.662792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37597a9-acec-488a-9ce0-968d11e43a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:32.663067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37597a9-acec-488a-9ce0-968d11e43a12 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:32.663731 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37597a9-acec-488a-9ce0-968d11e43a12 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:32.664064 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37597a9-acec-488a-9ce0-968d11e43a12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:32.665425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37597a9-acec-488a-9ce0-968d11e43a12 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:32.670164 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 637/1291] 10.222.0.78 () {56 vars in 1116 bytes} [Mon Sep 19 11:09:32 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:32.676035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c95d4f-d886-497a-90b6-1f18bb381322 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:32.778552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c95d4f-d886-497a-90b6-1f18bb381322 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:32.811812 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83594e1d-c7c4-4fd1-a4c4-961d27eca540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:32.812428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83594e1d-c7c4-4fd1-a4c4-961d27eca540 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:32.812706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83594e1d-c7c4-4fd1-a4c4-961d27eca540 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:32.812976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83594e1d-c7c4-4fd1-a4c4-961d27eca540 None admin] PATH_INFO: `/v3/services` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:32.813603 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83594e1d-c7c4-4fd1-a4c4-961d27eca540 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:32.813904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83594e1d-c7c4-4fd1-a4c4-961d27eca540 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:32.815281 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83594e1d-c7c4-4fd1-a4c4-961d27eca540 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:32.819701 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 655/1292] 10.222.0.78 () {56 vars in 1116 bytes} [Mon Sep 19 11:09:32 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:35.012329 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997de8b0-52df-47af-9639-8bd772e21286 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:35.012878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997de8b0-52df-47af-9639-8bd772e21286 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:35.012878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997de8b0-52df-47af-9639-8bd772e21286 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:35.014212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 638/1293] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:35.019723 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007e337-a744-44b7-b2fe-4dc319a268e5 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:35.020108 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007e337-a744-44b7-b2fe-4dc319a268e5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:35.020277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007e337-a744-44b7-b2fe-4dc319a268e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:35.039033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8007e337-a744-44b7-b2fe-4dc319a268e5 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:35.040290 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8007e337-a744-44b7-b2fe-4dc319a268e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:35.068809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:35.105286 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 656/1294] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:09:35.375490 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997de8b0-52df-47af-9639-8bd772e21286 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:35.407814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:35.478612 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997de8b0-52df-47af-9639-8bd772e21286 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:35.513640 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b2bd7-5d96-45db-a60a-d4f74e196243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:35.514243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b2bd7-5d96-45db-a60a-d4f74e196243 None admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:35.514517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b2bd7-5d96-45db-a60a-d4f74e196243 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:35.514786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b2bd7-5d96-45db-a60a-d4f74e196243 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:35.515465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b2bd7-5d96-45db-a60a-d4f74e196243 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:35.516813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b2bd7-5d96-45db-a60a-d4f74e196243 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:35.611838 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 639/1295] 10.222.0.78 () {60 vars in 1141 bytes} [Mon Sep 19 11:09:35 2022] POST /identity/v3/services => generated 246 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:09:37.825348 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707a7ac-b6e2-4ace-9146-c5ed6ff4bd43 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:37.825910 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707a7ac-b6e2-4ace-9146-c5ed6ff4bd43 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:37.825910 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707a7ac-b6e2-4ace-9146-c5ed6ff4bd43 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:37.827369 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 657/1296] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:37.832634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b375c6-cdf0-4b6b-a3ce-3e87ee9f03d8 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:37.833041 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b375c6-cdf0-4b6b-a3ce-3e87ee9f03d8 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:37.833178 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b375c6-cdf0-4b6b-a3ce-3e87ee9f03d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:37.851978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-f5b375c6-cdf0-4b6b-a3ce-3e87ee9f03d8 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:37.853228 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f5b375c6-cdf0-4b6b-a3ce-3e87ee9f03d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:37.928404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 640/1297] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:37 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:09:38.197517 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707a7ac-b6e2-4ace-9146-c5ed6ff4bd43 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:38.298497 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707a7ac-b6e2-4ace-9146-c5ed6ff4bd43 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:38.335632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14317b82-c7a1-48e4-afd0-72ac49b0d210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:38.336278 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14317b82-c7a1-48e4-afd0-72ac49b0d210 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:38.336563 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14317b82-c7a1-48e4-afd0-72ac49b0d210 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:38.336841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14317b82-c7a1-48e4-afd0-72ac49b0d210 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:38.337429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14317b82-c7a1-48e4-afd0-72ac49b0d210 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:38.338789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14317b82-c7a1-48e4-afd0-72ac49b0d210 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:38.347905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-14317b82-c7a1-48e4-afd0-72ac49b0d210 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:09:38.347905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:38.347905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:09:38.347905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:09:38.347905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:09:38.347905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:09:38.347905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:09:38.347905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:09:38.347905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:09:38.347905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:38.347905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:09:38.347905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:38.347905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:38.347905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:38.347905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:38.347905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:38.347905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:38.349181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:38.349181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:38.349181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:38.349181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:38.349181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:38.349181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:38.349181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:09:38.349181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:09:38.349181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:09:38.349181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:09:38.349181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:38.349181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:38.349181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:38.349181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:38.349181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:38.349181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:38.350054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:38.350054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:38.350054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:38.350054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:38.350054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:38.350054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:38.350054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:38.350054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:38.350054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:38.350054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:38.350054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:09:38.350054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:09:38.350054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:09:38.350054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:38.350054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 658/1298] 10.222.0.78 () {56 vars in 1117 bytes} [Mon Sep 19 11:09:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:38.355206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b375c6-cdf0-4b6b-a3ce-3e87ee9f03d8 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:38.374498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:38.458061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b375c6-cdf0-4b6b-a3ce-3e87ee9f03d8 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:38.491318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357f3bc-e2ea-46d1-ab0f-d7b5295db270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:38.491922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357f3bc-e2ea-46d1-ab0f-d7b5295db270 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:38.492202 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357f3bc-e2ea-46d1-ab0f-d7b5295db270 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:38.492482 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357f3bc-e2ea-46d1-ab0f-d7b5295db270 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:38.493364 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357f3bc-e2ea-46d1-ab0f-d7b5295db270 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:38.493666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357f3bc-e2ea-46d1-ab0f-d7b5295db270 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:38.495025 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357f3bc-e2ea-46d1-ab0f-d7b5295db270 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:38.499986 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 641/1299] 10.222.0.78 () {56 vars in 1116 bytes} [Mon Sep 19 11:09:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:38.506646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14317b82-c7a1-48e4-afd0-72ac49b0d210 None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:38.538214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:38.604270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14317b82-c7a1-48e4-afd0-72ac49b0d210 None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:38.639162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654a181-83d8-4f75-b0ed-1e95d83a9a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:38.639735 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654a181-83d8-4f75-b0ed-1e95d83a9a46 None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:38.640029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654a181-83d8-4f75-b0ed-1e95d83a9a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:38.640299 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654a181-83d8-4f75-b0ed-1e95d83a9a46 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:38.640994 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654a181-83d8-4f75-b0ed-1e95d83a9a46 None admin] RBAC: Adding query filter params (interface=public, service_id=6815a953ec5b46b4ba222c8c543e34bd, region_id=RegionOne) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:38.641301 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654a181-83d8-4f75-b0ed-1e95d83a9a46 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:38.642654 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654a181-83d8-4f75-b0ed-1e95d83a9a46 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:38.647837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 659/1300] 10.222.0.78 () {56 vars in 1251 bytes} [Mon Sep 19 11:09:38 2022] GET /identity/v3/endpoints?service_id=6815a953ec5b46b4ba222c8c543e34bd&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:38.653656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357f3bc-e2ea-46d1-ab0f-d7b5295db270 None admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:38.734425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:38.752668 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357f3bc-e2ea-46d1-ab0f-d7b5295db270 None admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:38.787006 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e46b0d-cf85-4852-a809-68c05e935111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:38.787645 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e46b0d-cf85-4852-a809-68c05e935111 None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:38.787924 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e46b0d-cf85-4852-a809-68c05e935111 None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:38.788196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e46b0d-cf85-4852-a809-68c05e935111 None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:38.788832 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e46b0d-cf85-4852-a809-68c05e935111 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:38.790189 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e46b0d-cf85-4852-a809-68c05e935111 None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:38.796957 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 642/1301] 10.222.0.78 () {56 vars in 1087 bytes} [Mon Sep 19 11:09:38 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:09:40.959417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f34c6-01d6-48a7-8f5e-f1c99967b80a None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:40.959958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f34c6-01d6-48a7-8f5e-f1c99967b80a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:40.959958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f34c6-01d6-48a7-8f5e-f1c99967b80a None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:40.961294 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 660/1302] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:40.966534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb5438-0744-4574-85b2-3891c1d4b44b None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:40.966972 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb5438-0744-4574-85b2-3891c1d4b44b None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:40.967096 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb5438-0744-4574-85b2-3891c1d4b44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:40.985246 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-4ebb5438-0744-4574-85b2-3891c1d4b44b None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:40.986432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4ebb5438-0744-4574-85b2-3891c1d4b44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:41.054601 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 643/1303] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:40 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:09:41.323745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f34c6-01d6-48a7-8f5e-f1c99967b80a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:41.422724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:41.430116 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f34c6-01d6-48a7-8f5e-f1c99967b80a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:41.464389 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0821db1-8005-4fb1-a23d-132a68fcaf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:41.465149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0821db1-8005-4fb1-a23d-132a68fcaf0e None admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:41.465427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0821db1-8005-4fb1-a23d-132a68fcaf0e None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:41.465698 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0821db1-8005-4fb1-a23d-132a68fcaf0e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:41.467575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0821db1-8005-4fb1-a23d-132a68fcaf0e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:41.467695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0821db1-8005-4fb1-a23d-132a68fcaf0e None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:41.476375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-a0821db1-8005-4fb1-a23d-132a68fcaf0e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:09:41.476375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:41.476375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:09:41.476375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:09:41.476375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:09:41.476375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:09:41.476375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:09:41.476375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:09:41.476375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:09:41.476375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:41.476375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:09:41.476375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:41.476375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:41.476375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:41.476375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:41.476375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:41.476375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:41.477575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:41.477575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:41.477575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:41.477575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:41.477575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:41.477575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:41.477575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:09:41.477575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:09:41.477575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:09:41.477575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:09:41.477575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:41.477575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:41.477575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:41.477575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:41.477575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:41.477575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:41.485578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:41.485578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:41.485578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:41.485578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:41.485578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:41.485578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:41.485578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:41.485578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:41.485578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:41.485578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:41.485578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:09:41.485578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:09:41.485578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:09:41.485578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:09:41.485578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 661/1304] 10.222.0.78 () {56 vars in 1117 bytes} [Mon Sep 19 11:09:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:41.486660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb5438-0744-4574-85b2-3891c1d4b44b None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:41.565350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:41.588763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb5438-0744-4574-85b2-3891c1d4b44b None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:41.622327 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b7344-dfc9-451e-b09b-2cdb067a7adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:41.622889 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b7344-dfc9-451e-b09b-2cdb067a7adb None admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:41.623163 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b7344-dfc9-451e-b09b-2cdb067a7adb None admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:41.623471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b7344-dfc9-451e-b09b-2cdb067a7adb None admin] PATH_INFO: `/v3/services` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:41.624585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b7344-dfc9-451e-b09b-2cdb067a7adb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:41.624585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b7344-dfc9-451e-b09b-2cdb067a7adb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:41.625794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b7344-dfc9-451e-b09b-2cdb067a7adb None admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:41.631679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 644/1305] 10.222.0.78 () {56 vars in 1116 bytes} [Mon Sep 19 11:09:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:41.639866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0821db1-8005-4fb1-a23d-132a68fcaf0e None admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:41.741602 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0821db1-8005-4fb1-a23d-132a68fcaf0e None admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:41.776117 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477cf5a-1fbc-459c-abd9-3475349af733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:41.776722 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477cf5a-1fbc-459c-abd9-3475349af733 None admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:41.777007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477cf5a-1fbc-459c-abd9-3475349af733 None admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:41.777291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477cf5a-1fbc-459c-abd9-3475349af733 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:41.777969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477cf5a-1fbc-459c-abd9-3475349af733 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:41.779358 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477cf5a-1fbc-459c-abd9-3475349af733 None admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:41.863775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 662/1306] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:09:41 2022] POST /identity/v3/endpoints => generated 331 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:09:51.633650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=30 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:51.858620 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:28.871474 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe53c90-ca83-4c29-9b57-e32b74147f01 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:28.871926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe53c90-ca83-4c29-9b57-e32b74147f01 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:28.872016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe53c90-ca83-4c29-9b57-e32b74147f01 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:28.879139 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3f30e-2c6f-4fb5-bb50-7dc3ad3ffb84 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:28.879570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3f30e-2c6f-4fb5-bb50-7dc3ad3ffb84 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:28.879740 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3f30e-2c6f-4fb5-bb50-7dc3ad3ffb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:28.883481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 645/1307] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:16:28 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:28.970340 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0ca3f30e-2c6f-4fb5-bb50-7dc3ad3ffb84 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:28.971711 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0ca3f30e-2c6f-4fb5-bb50-7dc3ad3ffb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:29.105493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 663/1308] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:16:29.415775 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a06c2-1e81-4197-9b73-a8b76aa25e43 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:29.416459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a06c2-1e81-4197-9b73-a8b76aa25e43 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:29.416969 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a06c2-1e81-4197-9b73-a8b76aa25e43 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:29.419026 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 646/1309] 10.222.0.78 () {54 vars in 963 bytes} [Mon Sep 19 11:16:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:29.428483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21392f-c4d9-4e22-8dc0-6b51fc8b2c33 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:29.428483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21392f-c4d9-4e22-8dc0-6b51fc8b2c33 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:29.428483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21392f-c4d9-4e22-8dc0-6b51fc8b2c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:29.528080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-bd21392f-c4d9-4e22-8dc0-6b51fc8b2c33 None None] MFA Rules not processed for user `dbf0049a3e3a442e945f6207477a7fd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:29.529338 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bd21392f-c4d9-4e22-8dc0-6b51fc8b2c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:29.553461 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:29.629546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 664/1310] 10.222.0.78 () {58 vars in 1060 bytes} [Mon Sep 19 11:16:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:29.638811 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a06c2-1e81-4197-9b73-a8b76aa25e43 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:29.640792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-405a06c2-1e81-4197-9b73-a8b76aa25e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:29.769146 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a06c2-1e81-4197-9b73-a8b76aa25e43 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:29.860770 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214be564-d940-4138-8f89-608fc3b6963f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:29.861391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214be564-d940-4138-8f89-608fc3b6963f service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:29.862044 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214be564-d940-4138-8f89-608fc3b6963f service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:29.862587 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214be564-d940-4138-8f89-608fc3b6963f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:29.864374 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-214be564-d940-4138-8f89-608fc3b6963f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:29.929712 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:29.996465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214be564-d940-4138-8f89-608fc3b6963f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:29.997908 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214be564-d940-4138-8f89-608fc3b6963f service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:30.018775 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 647/1311] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:32.532893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11482e87-b365-42eb-bab6-aac95a0ae99d None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:32.533417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11482e87-b365-42eb-bab6-aac95a0ae99d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:32.533531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11482e87-b365-42eb-bab6-aac95a0ae99d None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:32.534872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 665/1312] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:32.540251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4a273-0ae3-454e-a021-ccbe7106d7fd None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:32.540661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4a273-0ae3-454e-a021-ccbe7106d7fd None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:32.540807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4a273-0ae3-454e-a021-ccbe7106d7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:32.616475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-80c4a273-0ae3-454e-a021-ccbe7106d7fd None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:32.618551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-80c4a273-0ae3-454e-a021-ccbe7106d7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:32.723317 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 648/1313] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:33.025346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11482e87-b365-42eb-bab6-aac95a0ae99d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:33.026907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-11482e87-b365-42eb-bab6-aac95a0ae99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:33.135691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11482e87-b365-42eb-bab6-aac95a0ae99d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:33.208934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:33.211260 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c7408-a630-4fa9-84ab-4095cca8406c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:33.211866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c7408-a630-4fa9-84ab-4095cca8406c service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:33.212146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c7408-a630-4fa9-84ab-4095cca8406c service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:33.212416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c7408-a630-4fa9-84ab-4095cca8406c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:33.214012 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-478c7408-a630-4fa9-84ab-4095cca8406c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:33.321187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c7408-a630-4fa9-84ab-4095cca8406c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:33.322532 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c7408-a630-4fa9-84ab-4095cca8406c service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:33.340586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 666/1314] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:36.267376 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6d9bf-0da4-48f0-8406-48401c70ab0c None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:36.267990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6d9bf-0da4-48f0-8406-48401c70ab0c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:36.267990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6d9bf-0da4-48f0-8406-48401c70ab0c None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:36.269242 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 649/1315] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:36.274440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4403d-c33e-4d8f-99eb-6351ae3189af None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:36.274884 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4403d-c33e-4d8f-99eb-6351ae3189af None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:36.274987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4403d-c33e-4d8f-99eb-6351ae3189af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:36.338961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-77b4403d-c33e-4d8f-99eb-6351ae3189af None None] MFA Rules not processed for user `3cecb6cf43464e65ba7e69e9eb50ccbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:36.340195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-77b4403d-c33e-4d8f-99eb-6351ae3189af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:36.393116 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:36.438185 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 667/1316] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:36 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:36.729183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6d9bf-0da4-48f0-8406-48401c70ab0c None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:36.730749 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-3ed6d9bf-0da4-48f0-8406-48401c70ab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:36.741645 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:36.836312 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6d9bf-0da4-48f0-8406-48401c70ab0c None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:36.908943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad442a51-c376-421d-b465-3269a48a7a36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:36.909535 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad442a51-c376-421d-b465-3269a48a7a36 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:36.909807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad442a51-c376-421d-b465-3269a48a7a36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:36.910074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad442a51-c376-421d-b465-3269a48a7a36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:36.911670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ad442a51-c376-421d-b465-3269a48a7a36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:36.972264 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:37.019073 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad442a51-c376-421d-b465-3269a48a7a36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:37.020545 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad442a51-c376-421d-b465-3269a48a7a36 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:37.038779 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 650/1317] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:40.132804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382eb06-70bc-4352-89ff-4ade9f43ac75 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:40.133290 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382eb06-70bc-4352-89ff-4ade9f43ac75 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:40.133379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382eb06-70bc-4352-89ff-4ade9f43ac75 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:40.134675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 668/1318] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:40.140709 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ded659-d286-4bfb-801c-060c3b7c2d07 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:40.143401 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ded659-d286-4bfb-801c-060c3b7c2d07 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:40.143401 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ded659-d286-4bfb-801c-060c3b7c2d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:40.208305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-73ded659-d286-4bfb-801c-060c3b7c2d07 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:40.209521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-73ded659-d286-4bfb-801c-060c3b7c2d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:40.305274 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 651/1319] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:40.590634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382eb06-70bc-4352-89ff-4ade9f43ac75 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:40.592274 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8382eb06-70bc-4352-89ff-4ade9f43ac75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:40.714982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382eb06-70bc-4352-89ff-4ade9f43ac75 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:40.787730 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6414b6d-d2de-4cdf-b82a-4c1d37df2b6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:40.788339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6414b6d-d2de-4cdf-b82a-4c1d37df2b6d service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:40.788620 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6414b6d-d2de-4cdf-b82a-4c1d37df2b6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:40.788894 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6414b6d-d2de-4cdf-b82a-4c1d37df2b6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:40.790516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a6414b6d-d2de-4cdf-b82a-4c1d37df2b6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:40.797478 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:40.900018 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6414b6d-d2de-4cdf-b82a-4c1d37df2b6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:40.901367 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6414b6d-d2de-4cdf-b82a-4c1d37df2b6d service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:40.930288 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 669/1320] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:44.148317 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfa7ff-1b4f-4919-a6c1-22c27b9f522a None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:44.151846 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfa7ff-1b4f-4919-a6c1-22c27b9f522a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:44.151846 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfa7ff-1b4f-4919-a6c1-22c27b9f522a None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:44.151846 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 652/1321] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:44.155740 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d36da9-cbc5-410a-8949-2a9d0f09820f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:44.156168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d36da9-cbc5-410a-8949-2a9d0f09820f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:44.156291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d36da9-cbc5-410a-8949-2a9d0f09820f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:44.224100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-86d36da9-cbc5-410a-8949-2a9d0f09820f None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:44.225484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-86d36da9-cbc5-410a-8949-2a9d0f09820f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:44.285680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:44.325605 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 670/1322] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:44.616574 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfa7ff-1b4f-4919-a6c1-22c27b9f522a None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:44.618246 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c1bfa7ff-1b4f-4919-a6c1-22c27b9f522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:44.639633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:44.728085 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfa7ff-1b4f-4919-a6c1-22c27b9f522a None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:44.805510 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda36e7-4fd2-4e70-a560-9f21246a8676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:44.806113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda36e7-4fd2-4e70-a560-9f21246a8676 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:44.806393 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda36e7-4fd2-4e70-a560-9f21246a8676 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:44.806666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda36e7-4fd2-4e70-a560-9f21246a8676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:44.808323 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-8eda36e7-4fd2-4e70-a560-9f21246a8676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:44.880009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:44.919744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda36e7-4fd2-4e70-a560-9f21246a8676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:44.921680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda36e7-4fd2-4e70-a560-9f21246a8676 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:44.939627 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 653/1323] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:47.172761 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8ead9-d4cf-4579-963c-492d82d6081a None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:47.173323 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8ead9-d4cf-4579-963c-492d82d6081a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:47.173323 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8ead9-d4cf-4579-963c-492d82d6081a None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:47.174651 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 671/1324] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:47.180065 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb1575-4ee8-48d8-949d-47b13bac4bec None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:47.185357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb1575-4ee8-48d8-949d-47b13bac4bec None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:47.185357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb1575-4ee8-48d8-949d-47b13bac4bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:47.250274 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-2feb1575-4ee8-48d8-949d-47b13bac4bec None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:47.251531 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2feb1575-4ee8-48d8-949d-47b13bac4bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:47.355477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 654/1325] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:47.656068 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8ead9-d4cf-4579-963c-492d82d6081a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:47.664568 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-56b8ead9-d4cf-4579-963c-492d82d6081a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:47.776163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8ead9-d4cf-4579-963c-492d82d6081a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:47.850945 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929efa4d-fdb2-4049-963f-c0bf455d4c1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:47.851571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929efa4d-fdb2-4049-963f-c0bf455d4c1f service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:47.851851 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929efa4d-fdb2-4049-963f-c0bf455d4c1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:47.852119 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929efa4d-fdb2-4049-963f-c0bf455d4c1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:47.853715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-929efa4d-fdb2-4049-963f-c0bf455d4c1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:47.871391 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:47.963726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929efa4d-fdb2-4049-963f-c0bf455d4c1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:47.965115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929efa4d-fdb2-4049-963f-c0bf455d4c1f service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:47.983494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 672/1326] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:50.220878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d74e6e-62c1-4bc1-9fc2-4b21c3e0b2b9 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:50.221335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d74e6e-62c1-4bc1-9fc2-4b21c3e0b2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:50.221680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d74e6e-62c1-4bc1-9fc2-4b21c3e0b2b9 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:50.222788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 655/1327] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:50.228708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d2327-3422-4a53-93b9-3cbadc169415 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:50.228708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d2327-3422-4a53-93b9-3cbadc169415 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:50.229234 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d2327-3422-4a53-93b9-3cbadc169415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:50.304644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0a1d2327-3422-4a53-93b9-3cbadc169415 None None] MFA Rules not processed for user `3cecb6cf43464e65ba7e69e9eb50ccbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:50.305885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0a1d2327-3422-4a53-93b9-3cbadc169415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:50.374901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:50.406634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 673/1328] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:50 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:50.691012 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d74e6e-62c1-4bc1-9fc2-4b21c3e0b2b9 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:50.692983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-33d74e6e-62c1-4bc1-9fc2-4b21c3e0b2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:50.726229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:50.803443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d74e6e-62c1-4bc1-9fc2-4b21c3e0b2b9 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:50.887587 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad2ffd-f99e-4da1-b80a-2bb4364e925a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:50.888208 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad2ffd-f99e-4da1-b80a-2bb4364e925a service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:50.888486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad2ffd-f99e-4da1-b80a-2bb4364e925a service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:50.888754 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad2ffd-f99e-4da1-b80a-2bb4364e925a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:50.890962 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-04ad2ffd-f99e-4da1-b80a-2bb4364e925a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:50.979733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:51.019968 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad2ffd-f99e-4da1-b80a-2bb4364e925a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:51.021432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad2ffd-f99e-4da1-b80a-2bb4364e925a service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:51.048526 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 656/1329] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:50 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:53.806025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c11fe-69f7-4e1e-8eb0-931d8e321bf5 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:53.806660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c11fe-69f7-4e1e-8eb0-931d8e321bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:53.806660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c11fe-69f7-4e1e-8eb0-931d8e321bf5 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:53.807934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 674/1330] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:53.813754 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e29a5-9fd9-4478-96a0-ea306934c31f None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:53.814192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e29a5-9fd9-4478-96a0-ea306934c31f None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:53.814306 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e29a5-9fd9-4478-96a0-ea306934c31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:53.880901 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-ce7e29a5-9fd9-4478-96a0-ea306934c31f None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:53.882118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ce7e29a5-9fd9-4478-96a0-ea306934c31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:53.977838 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 657/1331] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:54.267894 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c11fe-69f7-4e1e-8eb0-931d8e321bf5 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:54.269438 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b65c11fe-69f7-4e1e-8eb0-931d8e321bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:54.370818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c11fe-69f7-4e1e-8eb0-931d8e321bf5 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:54.442668 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68068f-ac13-41ac-b05b-7b49edb6e9a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:54.443303 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68068f-ac13-41ac-b05b-7b49edb6e9a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:54.443579 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68068f-ac13-41ac-b05b-7b49edb6e9a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:54.443849 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68068f-ac13-41ac-b05b-7b49edb6e9a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:54.445489 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6f68068f-ac13-41ac-b05b-7b49edb6e9a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:54.472893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:54.557473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68068f-ac13-41ac-b05b-7b49edb6e9a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:54.558922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68068f-ac13-41ac-b05b-7b49edb6e9a7 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:54.577021 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 675/1332] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:56.808776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba880065-7a13-40f6-86b3-0b876d7a8b92 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:56.809335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba880065-7a13-40f6-86b3-0b876d7a8b92 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:56.809335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba880065-7a13-40f6-86b3-0b876d7a8b92 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:56.810716 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 658/1333] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:56.816530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff9f3c-4fa3-431b-99cd-c6205672c3f2 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:56.817022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff9f3c-4fa3-431b-99cd-c6205672c3f2 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:56.817134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff9f3c-4fa3-431b-99cd-c6205672c3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:56.892537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f7ff9f3c-4fa3-431b-99cd-c6205672c3f2 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:56.893704 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f7ff9f3c-4fa3-431b-99cd-c6205672c3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:56.978982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:57.002315 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 676/1334] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:57.290964 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba880065-7a13-40f6-86b3-0b876d7a8b92 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:57.292524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ba880065-7a13-40f6-86b3-0b876d7a8b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:57.333258 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:57.395063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba880065-7a13-40f6-86b3-0b876d7a8b92 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:57.469253 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c132d1-446f-4ab0-ab8b-350dca09c6e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:57.469868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c132d1-446f-4ab0-ab8b-350dca09c6e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:57.470149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c132d1-446f-4ab0-ab8b-350dca09c6e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:57.470424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c132d1-446f-4ab0-ab8b-350dca09c6e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:57.472059 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-06c132d1-446f-4ab0-ab8b-350dca09c6e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:57.560091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:57.586329 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c132d1-446f-4ab0-ab8b-350dca09c6e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:57.587795 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c132d1-446f-4ab0-ab8b-350dca09c6e7 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:57.605987 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 659/1335] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:00.561439 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72504afb-abf9-4931-a191-dab105269849 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:00.562003 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72504afb-abf9-4931-a191-dab105269849 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:00.562003 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72504afb-abf9-4931-a191-dab105269849 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:00.563353 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 677/1336] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:00.568552 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28cad3-26da-4da5-9571-2030e864a87c None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:00.568997 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28cad3-26da-4da5-9571-2030e864a87c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:00.569105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28cad3-26da-4da5-9571-2030e864a87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:00.634097 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-2c28cad3-26da-4da5-9571-2030e864a87c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:00.635317 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2c28cad3-26da-4da5-9571-2030e864a87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:00.729270 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 660/1337] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:17:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:01.010075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72504afb-abf9-4931-a191-dab105269849 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:01.011851 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-72504afb-abf9-4931-a191-dab105269849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:01.123640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72504afb-abf9-4931-a191-dab105269849 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:01.196686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d96b8-29e9-4090-a9aa-8023555487f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:01.197362 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d96b8-29e9-4090-a9aa-8023555487f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:01.197650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d96b8-29e9-4090-a9aa-8023555487f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:01.197926 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d96b8-29e9-4090-a9aa-8023555487f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:01.199608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ac6d96b8-29e9-4090-a9aa-8023555487f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:01.237171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:01.312380 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d96b8-29e9-4090-a9aa-8023555487f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:01.313919 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d96b8-29e9-4090-a9aa-8023555487f3 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:01.332809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 678/1338] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:05.655093 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f73c7d-0a2d-42f7-b18d-238b5159a51d None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:05.655758 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f73c7d-0a2d-42f7-b18d-238b5159a51d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:05.655758 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f73c7d-0a2d-42f7-b18d-238b5159a51d None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:05.656997 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 661/1339] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:05.663136 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da41c4-78c8-41a6-bca7-131dd8be6141 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:05.663621 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da41c4-78c8-41a6-bca7-131dd8be6141 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:05.663734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da41c4-78c8-41a6-bca7-131dd8be6141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:05.730404 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f1da41c4-78c8-41a6-bca7-131dd8be6141 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:05.731663 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f1da41c4-78c8-41a6-bca7-131dd8be6141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:05.827512 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:05.838165 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 679/1340] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:06.120867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f73c7d-0a2d-42f7-b18d-238b5159a51d None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:06.122517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-63f73c7d-0a2d-42f7-b18d-238b5159a51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:06.177955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:06.242571 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f73c7d-0a2d-42f7-b18d-238b5159a51d None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:06.325345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4dcfa3-9135-40a7-94e8-60e68248f69b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:06.325985 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dcfa3-9135-40a7-94e8-60e68248f69b service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:06.326266 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dcfa3-9135-40a7-94e8-60e68248f69b service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:06.326541 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dcfa3-9135-40a7-94e8-60e68248f69b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:06.328222 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1f4dcfa3-9135-40a7-94e8-60e68248f69b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:06.427700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:06.454055 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4dcfa3-9135-40a7-94e8-60e68248f69b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:06.454055 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4dcfa3-9135-40a7-94e8-60e68248f69b service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:06.464954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 662/1341] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:17:09.649318 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64759737-5094-411a-8b88-08b9915b5f35 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:09.649907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64759737-5094-411a-8b88-08b9915b5f35 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:09.649907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64759737-5094-411a-8b88-08b9915b5f35 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:09.651231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 680/1342] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:09.656447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3b5d7-de57-4cf5-9af5-59d9a4d3099b None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:09.656896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3b5d7-de57-4cf5-9af5-59d9a4d3099b None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:09.656991 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3b5d7-de57-4cf5-9af5-59d9a4d3099b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:09.724945 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-32e3b5d7-de57-4cf5-9af5-59d9a4d3099b None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:09.726180 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-32e3b5d7-de57-4cf5-9af5-59d9a4d3099b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:09.824867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 663/1343] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:17:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:10.121374 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64759737-5094-411a-8b88-08b9915b5f35 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:10.123064 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-64759737-5094-411a-8b88-08b9915b5f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:10.234125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64759737-5094-411a-8b88-08b9915b5f35 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:10.309039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1c825-b18d-4bb4-aa55-73cf0e3fbf24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:10.309570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1c825-b18d-4bb4-aa55-73cf0e3fbf24 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:10.309849 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1c825-b18d-4bb4-aa55-73cf0e3fbf24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:10.310121 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1c825-b18d-4bb4-aa55-73cf0e3fbf24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:10.311786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-37d1c825-b18d-4bb4-aa55-73cf0e3fbf24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:10.363414 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:10.443202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d1c825-b18d-4bb4-aa55-73cf0e3fbf24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:10.444719 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d1c825-b18d-4bb4-aa55-73cf0e3fbf24 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:10.463684 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 681/1344] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:15.175004 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65c098-0c13-41fe-8368-569019b661c4 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:15.175610 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65c098-0c13-41fe-8368-569019b661c4 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:15.175610 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65c098-0c13-41fe-8368-569019b661c4 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:15.177032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 664/1345] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:15.182438 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0c655-3d9d-4022-b237-e3979760b114 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:15.195953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0c655-3d9d-4022-b237-e3979760b114 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:15.195953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0c655-3d9d-4022-b237-e3979760b114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:15.252997 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-16b0c655-3d9d-4022-b237-e3979760b114 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:15.254256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-16b0c655-3d9d-4022-b237-e3979760b114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:15.356789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:15.361907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 682/1346] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:15.658854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97e017-5b06-4d9a-bc68-f3cb4eab2f58 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:15.659140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97e017-5b06-4d9a-bc68-f3cb4eab2f58 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:15.659445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97e017-5b06-4d9a-bc68-f3cb4eab2f58 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:15.660795 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 665/1347] 10.222.0.78 () {54 vars in 963 bytes} [Mon Sep 19 11:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:15.666567 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110bbe9-91c2-408e-85b5-5d0dc449a634 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:15.666852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110bbe9-91c2-408e-85b5-5d0dc449a634 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:15.667128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110bbe9-91c2-408e-85b5-5d0dc449a634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:15.743083 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-2110bbe9-91c2-408e-85b5-5d0dc449a634 None None] MFA Rules not processed for user `dbf0049a3e3a442e945f6207477a7fd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:15.744337 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2110bbe9-91c2-408e-85b5-5d0dc449a634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:15.820421 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 683/1348] 10.222.0.78 () {58 vars in 1060 bytes} [Mon Sep 19 11:17:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:15.829620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97e017-5b06-4d9a-bc68-f3cb4eab2f58 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:15.831189 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-cb97e017-5b06-4d9a-bc68-f3cb4eab2f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:15.888986 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:15.938709 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97e017-5b06-4d9a-bc68-f3cb4eab2f58 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:16.013869 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0c13a-7623-4e60-95fe-a5f717f7f46d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:16.014581 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0c13a-7623-4e60-95fe-a5f717f7f46d service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:16.014862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0c13a-7623-4e60-95fe-a5f717f7f46d service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:16.015137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0c13a-7623-4e60-95fe-a5f717f7f46d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:16.016774 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e8b0c13a-7623-4e60-95fe-a5f717f7f46d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:16.131249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:16.134857 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0c13a-7623-4e60-95fe-a5f717f7f46d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:16.136324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0c13a-7623-4e60-95fe-a5f717f7f46d service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:16.155368 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 666/1349] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:21.572627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e06e97-597e-4790-bdec-1d6b57977d96 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:21.573612 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e06e97-597e-4790-bdec-1d6b57977d96 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:21.574168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e06e97-597e-4790-bdec-1d6b57977d96 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:21.577470 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 684/1350] 10.222.0.78 () {54 vars in 921 bytes} [Mon Sep 19 11:17:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:21.587964 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de6be6-a0de-466c-9f7e-7244d70c8ba0 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:21.587964 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de6be6-a0de-466c-9f7e-7244d70c8ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:21.587964 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de6be6-a0de-466c-9f7e-7244d70c8ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:21.688114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-b9de6be6-a0de-466c-9f7e-7244d70c8ba0 None None] MFA Rules not processed for user `0427cdd3905a422cab4fed0aac187ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:21.689335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b9de6be6-a0de-466c-9f7e-7244d70c8ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:21.818707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 667/1351] 10.222.0.78 () {58 vars in 1018 bytes} [Mon Sep 19 11:17:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:25.818886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=59 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:26.008932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96f23c-e20b-4467-b8ae-d8d392131618 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:26.009709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96f23c-e20b-4467-b8ae-d8d392131618 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:26.010197 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96f23c-e20b-4467-b8ae-d8d392131618 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:26.011804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 685/1352] 10.222.0.78 () {54 vars in 921 bytes} [Mon Sep 19 11:17:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:26.018136 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d78330-8e3f-4627-aef2-274cbe891b0b None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:26.018659 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d78330-8e3f-4627-aef2-274cbe891b0b None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:26.018763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d78330-8e3f-4627-aef2-274cbe891b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:26.081809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:26.117348 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-a4d78330-8e3f-4627-aef2-274cbe891b0b None None] MFA Rules not processed for user `0427cdd3905a422cab4fed0aac187ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:26.147918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a4d78330-8e3f-4627-aef2-274cbe891b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:26.237554 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 668/1353] 10.222.0.78 () {58 vars in 1018 bytes} [Mon Sep 19 11:17:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:27.826707 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1382d4-6be5-4a29-b869-d23f241971ee None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:27.827220 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1382d4-6be5-4a29-b869-d23f241971ee None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:27.827220 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1382d4-6be5-4a29-b869-d23f241971ee None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:27.828514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 686/1354] 10.222.0.78 () {54 vars in 921 bytes} [Mon Sep 19 11:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:27.833644 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28ede9-ef24-4687-bee0-c1d06061be4a None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:27.834084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28ede9-ef24-4687-bee0-c1d06061be4a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:27.834325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28ede9-ef24-4687-bee0-c1d06061be4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:27.918654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-1f28ede9-ef24-4687-bee0-c1d06061be4a None None] MFA Rules not processed for user `0427cdd3905a422cab4fed0aac187ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:27.919963 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1f28ede9-ef24-4687-bee0-c1d06061be4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:27.963511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:28.036575 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 669/1355] 10.222.0.78 () {58 vars in 1018 bytes} [Mon Sep 19 11:17:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:31.515322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bb887-dcfa-4841-9952-140aa3058687 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:31.516170 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bb887-dcfa-4841-9952-140aa3058687 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:31.516626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bb887-dcfa-4841-9952-140aa3058687 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:31.518450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 687/1356] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:17:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:31.524307 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7deede-d50f-481d-95f3-61fafcfb905e None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:31.525489 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7deede-d50f-481d-95f3-61fafcfb905e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:31.526002 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7deede-d50f-481d-95f3-61fafcfb905e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:31.604016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-ec7deede-d50f-481d-95f3-61fafcfb905e None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:31.605378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ec7deede-d50f-481d-95f3-61fafcfb905e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:31.709660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 670/1357] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:32.014521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1ab77-25f5-48cd-9f91-86fe743180df None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:32.015276 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1ab77-25f5-48cd-9f91-86fe743180df None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:32.015763 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1ab77-25f5-48cd-9f91-86fe743180df None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:32.017924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 688/1358] 10.222.0.78 () {54 vars in 965 bytes} [Mon Sep 19 11:17:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:32.030356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29245c85-183d-4ae1-b81e-f498d3ed8572 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:32.031101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29245c85-183d-4ae1-b81e-f498d3ed8572 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:32.031650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29245c85-183d-4ae1-b81e-f498d3ed8572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:32.056283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:32.153508 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-29245c85-183d-4ae1-b81e-f498d3ed8572 None None] MFA Rules not processed for user `aba1968529574fe5b4c5d114c874f428`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:32.156070 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-29245c85-183d-4ae1-b81e-f498d3ed8572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:32.283601 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 671/1359] 10.222.0.78 () {58 vars in 1062 bytes} [Mon Sep 19 11:17:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:17:32.291445 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1ab77-25f5-48cd-9f91-86fe743180df None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:32.292970 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2ae1ab77-25f5-48cd-9f91-86fe743180df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:32.568457 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1ab77-25f5-48cd-9f91-86fe743180df None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:32.656905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:32.701445 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ea4dc-6e63-47f2-aaf3-4dca6db4db44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:32.702066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ea4dc-6e63-47f2-aaf3-4dca6db4db44 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:32.702350 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ea4dc-6e63-47f2-aaf3-4dca6db4db44 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:32.702627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ea4dc-6e63-47f2-aaf3-4dca6db4db44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:32.704294 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b92ea4dc-6e63-47f2-aaf3-4dca6db4db44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:32.804471 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ea4dc-6e63-47f2-aaf3-4dca6db4db44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:32.806078 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ea4dc-6e63-47f2-aaf3-4dca6db4db44 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:32.826950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 689/1360] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:35.246878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f03956-a04f-4ef4-ad30-15885dcad720 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:35.246878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f03956-a04f-4ef4-ad30-15885dcad720 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:35.246878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f03956-a04f-4ef4-ad30-15885dcad720 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:35.251885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 672/1361] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:17:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:35.254057 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896407e-45fd-4858-9030-c86fdfaae384 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:35.254913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896407e-45fd-4858-9030-c86fdfaae384 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:35.255395 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896407e-45fd-4858-9030-c86fdfaae384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:35.328937 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c896407e-45fd-4858-9030-c86fdfaae384 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:35.330305 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c896407e-45fd-4858-9030-c86fdfaae384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:35.373962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:35.440626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 690/1362] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:35.731514 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f03956-a04f-4ef4-ad30-15885dcad720 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:35.733865 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a6f03956-a04f-4ef4-ad30-15885dcad720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:35.827836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:35.925670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f03956-a04f-4ef4-ad30-15885dcad720 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:36.046951 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466e1f3-9b3b-443b-b231-b94f2481a375 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:36.047594 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466e1f3-9b3b-443b-b231-b94f2481a375 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:36.047870 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466e1f3-9b3b-443b-b231-b94f2481a375 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:36.048139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466e1f3-9b3b-443b-b231-b94f2481a375 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:36.049751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2466e1f3-9b3b-443b-b231-b94f2481a375 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:36.074279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:36.156029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466e1f3-9b3b-443b-b231-b94f2481a375 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:36.157648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466e1f3-9b3b-443b-b231-b94f2481a375 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:36.181112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 673/1363] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:38.914497 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7ef0a-9020-412c-8b38-f6124e7aba6e None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:38.915057 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7ef0a-9020-412c-8b38-f6124e7aba6e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:38.915057 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7ef0a-9020-412c-8b38-f6124e7aba6e None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:38.919633 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 691/1364] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:38.922357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908abcb6-1711-4341-a896-13342302d623 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:38.922793 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908abcb6-1711-4341-a896-13342302d623 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:38.923075 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908abcb6-1711-4341-a896-13342302d623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:38.998627 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-908abcb6-1711-4341-a896-13342302d623 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:39.000231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-908abcb6-1711-4341-a896-13342302d623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:39.117856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 674/1365] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:39.120335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:39.406193 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b6fb8-ebdb-402c-9774-975a69382f1a None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:39.406497 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b6fb8-ebdb-402c-9774-975a69382f1a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:39.406899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b6fb8-ebdb-402c-9774-975a69382f1a None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:39.408532 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 692/1366] 10.222.0.78 () {54 vars in 965 bytes} [Mon Sep 19 11:17:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:39.414348 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed92f8-c6c2-48fd-8945-594b729b625e None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:39.422743 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed92f8-c6c2-48fd-8945-594b729b625e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:39.422743 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed92f8-c6c2-48fd-8945-594b729b625e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:39.521252 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-55ed92f8-c6c2-48fd-8945-594b729b625e None None] MFA Rules not processed for user `aba1968529574fe5b4c5d114c874f428`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:39.523009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-55ed92f8-c6c2-48fd-8945-594b729b625e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:39.660062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 675/1367] 10.222.0.78 () {58 vars in 1062 bytes} [Mon Sep 19 11:17:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:39.668831 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b6fb8-ebdb-402c-9774-975a69382f1a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:39.671364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d75b6fb8-ebdb-402c-9774-975a69382f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:39.792045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e18a8-abd7-4dea-a772-53837100f710 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:39.792510 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e18a8-abd7-4dea-a772-53837100f710 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:39.792897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e18a8-abd7-4dea-a772-53837100f710 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:39.794621 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 676/1368] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:17:39.800684 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e9805-68c8-4e87-8abb-68a0b5fba29d None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:39.801429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e9805-68c8-4e87-8abb-68a0b5fba29d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:39.801915 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e9805-68c8-4e87-8abb-68a0b5fba29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:39.839386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b6fb8-ebdb-402c-9774-975a69382f1a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:39.863449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:39.877860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:39.886586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-c17e9805-68c8-4e87-8abb-68a0b5fba29d None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:39.888755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c17e9805-68c8-4e87-8abb-68a0b5fba29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:39.977706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02658be1-69fd-4d75-9644-be26cf22d7f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:39.978281 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02658be1-69fd-4d75-9644-be26cf22d7f2 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:39.978742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02658be1-69fd-4d75-9644-be26cf22d7f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:39.978742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02658be1-69fd-4d75-9644-be26cf22d7f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:39.980529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-02658be1-69fd-4d75-9644-be26cf22d7f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:40.004258 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 677/1369] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:17:40.078120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02658be1-69fd-4d75-9644-be26cf22d7f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:40.079765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02658be1-69fd-4d75-9644-be26cf22d7f2 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:40.099142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 693/1370] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:40.344550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d65cf2-1665-4418-b408-51b4566ed62d None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:40.344838 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d65cf2-1665-4418-b408-51b4566ed62d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:40.345110 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d65cf2-1665-4418-b408-51b4566ed62d None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:40.346624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 678/1371] 10.222.0.78 () {54 vars in 967 bytes} [Mon Sep 19 11:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:40.351777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4754b9-0e07-4b30-9ab7-7b2af57a1924 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:40.352069 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4754b9-0e07-4b30-9ab7-7b2af57a1924 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:40.361910 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4754b9-0e07-4b30-9ab7-7b2af57a1924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:40.391134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:40.447809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8d4754b9-0e07-4b30-9ab7-7b2af57a1924 None None] MFA Rules not processed for user `1ab94f6e08644fa7a76b9057e46d122d`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:40.449079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8d4754b9-0e07-4b30-9ab7-7b2af57a1924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:40.547890 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 694/1372] 10.222.0.78 () {58 vars in 1064 bytes} [Mon Sep 19 11:17:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:40.555583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d65cf2-1665-4418-b408-51b4566ed62d None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:40.557248 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-21d65cf2-1665-4418-b408-51b4566ed62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:40.671183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d65cf2-1665-4418-b408-51b4566ed62d None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:40.703279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:40.761583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf365d-7820-4c62-9ddb-ee4c6ca9a581 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:40.762209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf365d-7820-4c62-9ddb-ee4c6ca9a581 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:40.762464 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf365d-7820-4c62-9ddb-ee4c6ca9a581 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:40.762772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf365d-7820-4c62-9ddb-ee4c6ca9a581 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:40.764435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-bfbf365d-7820-4c62-9ddb-ee4c6ca9a581 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:40.884951 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf365d-7820-4c62-9ddb-ee4c6ca9a581 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:40.886358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf365d-7820-4c62-9ddb-ee4c6ca9a581 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:40.908881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 679/1373] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:17:42.536917 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c70e4-02a9-4e20-9850-95f9250e173f None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:42.537474 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c70e4-02a9-4e20-9850-95f9250e173f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:42.537474 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c70e4-02a9-4e20-9850-95f9250e173f None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:42.538770 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 695/1374] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:42.544063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbf34b-19ec-4552-aa74-8de6d8ffed2e None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:42.544489 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbf34b-19ec-4552-aa74-8de6d8ffed2e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:42.544737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbf34b-19ec-4552-aa74-8de6d8ffed2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:42.625486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-5cdbf34b-19ec-4552-aa74-8de6d8ffed2e None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:42.626732 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5cdbf34b-19ec-4552-aa74-8de6d8ffed2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:42.633730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:42.723888 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 680/1375] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:43.022213 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c70e4-02a9-4e20-9850-95f9250e173f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:43.023804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bd4c70e4-02a9-4e20-9850-95f9250e173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:43.152070 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:43.192265 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c70e4-02a9-4e20-9850-95f9250e173f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:43.323028 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e928f58-0e71-4ad4-a3e4-4d06c2f5cbed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:43.323788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e928f58-0e71-4ad4-a3e4-4d06c2f5cbed service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:43.324073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e928f58-0e71-4ad4-a3e4-4d06c2f5cbed service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:43.324347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e928f58-0e71-4ad4-a3e4-4d06c2f5cbed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:43.326040 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6e928f58-0e71-4ad4-a3e4-4d06c2f5cbed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:43.409797 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:43.424486 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e928f58-0e71-4ad4-a3e4-4d06c2f5cbed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:43.426410 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e928f58-0e71-4ad4-a3e4-4d06c2f5cbed service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:43.452585 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 696/1376] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:17:45.920748 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71837f1-a618-4de1-8975-89a21c889a3b None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:45.921659 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71837f1-a618-4de1-8975-89a21c889a3b None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:45.921939 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71837f1-a618-4de1-8975-89a21c889a3b None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:45.923476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 681/1377] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:17:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:45.928599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c2eb0-4c58-44a6-8ae3-bf953c34c5cb None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:45.929048 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c2eb0-4c58-44a6-8ae3-bf953c34c5cb None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:45.929151 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c2eb0-4c58-44a6-8ae3-bf953c34c5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:45.999830 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-fd3c2eb0-4c58-44a6-8ae3-bf953c34c5cb None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:46.001062 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fd3c2eb0-4c58-44a6-8ae3-bf953c34c5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:46.119332 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 697/1378] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:46.408695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71837f1-a618-4de1-8975-89a21c889a3b None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:46.410885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d71837f1-a618-4de1-8975-89a21c889a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:46.566061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:46.577673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71837f1-a618-4de1-8975-89a21c889a3b None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:46.705635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917a836-daf8-4abc-a6d7-e949035f07f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:46.706154 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917a836-daf8-4abc-a6d7-e949035f07f0 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:46.706431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917a836-daf8-4abc-a6d7-e949035f07f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:46.706703 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917a836-daf8-4abc-a6d7-e949035f07f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:46.708355 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-9917a836-daf8-4abc-a6d7-e949035f07f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:46.814864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917a836-daf8-4abc-a6d7-e949035f07f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:46.816991 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917a836-daf8-4abc-a6d7-e949035f07f0 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:46.829574 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:46.838472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 682/1379] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:49.229730 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a367ea4-9fb6-41a0-94e7-f95ba1ee8bd3 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:49.230276 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a367ea4-9fb6-41a0-94e7-f95ba1ee8bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:49.230407 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a367ea4-9fb6-41a0-94e7-f95ba1ee8bd3 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:49.232421 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 698/1380] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:17:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:49.238919 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691818c-3f3d-4c7c-81d9-6d5a2b95f03c None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:49.239564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691818c-3f3d-4c7c-81d9-6d5a2b95f03c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:49.239564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691818c-3f3d-4c7c-81d9-6d5a2b95f03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:49.310929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-3691818c-3f3d-4c7c-81d9-6d5a2b95f03c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:49.312557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-3691818c-3f3d-4c7c-81d9-6d5a2b95f03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:49.438960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 683/1381] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:17:49.745315 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a367ea4-9fb6-41a0-94e7-f95ba1ee8bd3 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:49.747467 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5a367ea4-9fb6-41a0-94e7-f95ba1ee8bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:49.823693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:49.913783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a367ea4-9fb6-41a0-94e7-f95ba1ee8bd3 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:50.030081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934196d0-db17-4026-84b5-8e25f2037684 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:50.030703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934196d0-db17-4026-84b5-8e25f2037684 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:50.030984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934196d0-db17-4026-84b5-8e25f2037684 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:50.031334 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934196d0-db17-4026-84b5-8e25f2037684 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:50.032983 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-934196d0-db17-4026-84b5-8e25f2037684 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:50.058871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:50.146088 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934196d0-db17-4026-84b5-8e25f2037684 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:50.147726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934196d0-db17-4026-84b5-8e25f2037684 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:50.167654 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 699/1382] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:52.860593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966c323-bde2-4c37-b6a3-59c84f8d6a4a None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:52.861211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966c323-bde2-4c37-b6a3-59c84f8d6a4a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:52.861326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966c323-bde2-4c37-b6a3-59c84f8d6a4a None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:52.863100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 684/1383] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:17:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:52.868895 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36defdac-7da4-43a6-88a8-d01d0988f9e8 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:52.869377 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36defdac-7da4-43a6-88a8-d01d0988f9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:52.869493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36defdac-7da4-43a6-88a8-d01d0988f9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:52.953001 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-36defdac-7da4-43a6-88a8-d01d0988f9e8 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:52.954535 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-36defdac-7da4-43a6-88a8-d01d0988f9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:53.066763 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:53.070599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 700/1384] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:53.365810 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966c323-bde2-4c37-b6a3-59c84f8d6a4a None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:53.367959 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2966c323-bde2-4c37-b6a3-59c84f8d6a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:53.464737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:53.559475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966c323-bde2-4c37-b6a3-59c84f8d6a4a None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:53.693783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93aa7b-477e-4b42-8b8d-e1b93ff496f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:53.694519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93aa7b-477e-4b42-8b8d-e1b93ff496f2 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:53.694911 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93aa7b-477e-4b42-8b8d-e1b93ff496f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:53.695278 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93aa7b-477e-4b42-8b8d-e1b93ff496f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:53.697361 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2b93aa7b-477e-4b42-8b8d-e1b93ff496f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:53.744794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:53.814321 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93aa7b-477e-4b42-8b8d-e1b93ff496f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:53.815979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93aa7b-477e-4b42-8b8d-e1b93ff496f2 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:53.835808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 685/1385] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:56.028817 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bf64c-3073-4c7b-bbd0-00bcbe85242c None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:56.029912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bf64c-3073-4c7b-bbd0-00bcbe85242c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:56.029912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bf64c-3073-4c7b-bbd0-00bcbe85242c None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:56.031303 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 701/1386] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:17:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:56.036875 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238650d4-3999-4bb8-9ec1-01bb884e00e8 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:56.037304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238650d4-3999-4bb8-9ec1-01bb884e00e8 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:56.037408 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238650d4-3999-4bb8-9ec1-01bb884e00e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:56.105079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-238650d4-3999-4bb8-9ec1-01bb884e00e8 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:56.106306 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-238650d4-3999-4bb8-9ec1-01bb884e00e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:56.218611 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 686/1387] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:56.524724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7bf64c-3073-4c7b-bbd0-00bcbe85242c None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:56.526259 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0d7bf64c-3073-4c7b-bbd0-00bcbe85242c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:56.679066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7bf64c-3073-4c7b-bbd0-00bcbe85242c None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:56.747047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:56.799214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148506a-5f41-444c-8a62-3df976ad0067 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:56.799867 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148506a-5f41-444c-8a62-3df976ad0067 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:56.800153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148506a-5f41-444c-8a62-3df976ad0067 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:56.800424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148506a-5f41-444c-8a62-3df976ad0067 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:56.802132 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b148506a-5f41-444c-8a62-3df976ad0067 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:56.895009 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148506a-5f41-444c-8a62-3df976ad0067 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:56.896648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148506a-5f41-444c-8a62-3df976ad0067 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:56.914548 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 702/1388] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:57.318514 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cd5ba-04a0-4646-8fc2-428a742ddda6 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:57.319096 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cd5ba-04a0-4646-8fc2-428a742ddda6 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:57.319096 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cd5ba-04a0-4646-8fc2-428a742ddda6 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:57.320492 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 687/1389] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:57.325943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd1f2c-217c-4a86-9401-2b87a2339a61 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:57.326382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd1f2c-217c-4a86-9401-2b87a2339a61 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:57.326632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd1f2c-217c-4a86-9401-2b87a2339a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:57.395368 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-12dd1f2c-217c-4a86-9401-2b87a2339a61 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:57.396495 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-12dd1f2c-217c-4a86-9401-2b87a2339a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:57.432511 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:57.511662 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 703/1390] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:57.808884 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765cfe2-4a33-43ae-9141-38beda07804a None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:57.808884 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765cfe2-4a33-43ae-9141-38beda07804a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:57.808884 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765cfe2-4a33-43ae-9141-38beda07804a None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:57.809893 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 688/1391] 10.222.0.78 () {54 vars in 962 bytes} [Mon Sep 19 11:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:57.816509 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6aafb9-2231-4792-9932-b2f5695c4c96 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:57.816801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6aafb9-2231-4792-9932-b2f5695c4c96 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:57.817081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6aafb9-2231-4792-9932-b2f5695c4c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:57.905299 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-7a6aafb9-2231-4792-9932-b2f5695c4c96 None None] MFA Rules not processed for user `558e3b6dd9754c57ad028aa431a68b80`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:57.906520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7a6aafb9-2231-4792-9932-b2f5695c4c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:57.986778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 704/1392] 10.222.0.78 () {58 vars in 1059 bytes} [Mon Sep 19 11:17:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:57.995413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765cfe2-4a33-43ae-9141-38beda07804a None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:57.996928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1765cfe2-4a33-43ae-9141-38beda07804a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:58.011728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:58.103087 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765cfe2-4a33-43ae-9141-38beda07804a None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:58.192601 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a353c-9499-434f-89c1-6cfdfafca601 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558e3b6dd9754c57ad028aa431a68b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:58.193216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a353c-9499-434f-89c1-6cfdfafca601 service glance] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:58.193495 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a353c-9499-434f-89c1-6cfdfafca601 service glance] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:58.193768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a353c-9499-434f-89c1-6cfdfafca601 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:58.195426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c44a353c-9499-434f-89c1-6cfdfafca601 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:58.267210 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:58.309341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a353c-9499-434f-89c1-6cfdfafca601 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:58.310782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a353c-9499-434f-89c1-6cfdfafca601 service glance] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:58.333419 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 689/1393] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:59.282680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a9934-a58e-48f8-b4fb-159f97256dcb None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:59.283230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a9934-a58e-48f8-b4fb-159f97256dcb None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:59.283342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a9934-a58e-48f8-b4fb-159f97256dcb None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:59.284739 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 705/1394] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:59.290157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d61026-2aaf-4f38-ad7e-4ca3c964fe73 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:59.290447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d61026-2aaf-4f38-ad7e-4ca3c964fe73 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:59.290722 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d61026-2aaf-4f38-ad7e-4ca3c964fe73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:59.309203 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cacdb-b8b7-47d8-8e7b-3df2bdba31b1 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:59.309569 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cacdb-b8b7-47d8-8e7b-3df2bdba31b1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:59.309918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cacdb-b8b7-47d8-8e7b-3df2bdba31b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:59.347187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:59.361534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-34d61026-2aaf-4f38-ad7e-4ca3c964fe73 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:59.362792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-34d61026-2aaf-4f38-ad7e-4ca3c964fe73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:59.380396 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-2c2cacdb-b8b7-47d8-8e7b-3df2bdba31b1 None None] MFA Rules not processed for user `5a667fa07b1148998959835d9995ad60`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:59.381544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2c2cacdb-b8b7-47d8-8e7b-3df2bdba31b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:59.469557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 690/1395] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:59.474195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 706/1396] 10.222.0.78 () {58 vars in 1009 bytes} [Mon Sep 19 11:17:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:17:59.520038 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4044b5-cf74-4b73-8a37-267447d98e0f None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:59.520413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4044b5-cf74-4b73-8a37-267447d98e0f None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:59.520760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4044b5-cf74-4b73-8a37-267447d98e0f None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:59.522197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 691/1397] 10.222.0.78 () {54 vars in 943 bytes} [Mon Sep 19 11:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:59.529287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e218a-38dc-4014-9d01-0c5fa85aaafe None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:59.529588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e218a-38dc-4014-9d01-0c5fa85aaafe None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:59.529867 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e218a-38dc-4014-9d01-0c5fa85aaafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:59.614187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b61e218a-38dc-4014-9d01-0c5fa85aaafe None None] MFA Rules not processed for user `cf462b2aa3fc4e2ca58851549cf0c644`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:59.615431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b61e218a-38dc-4014-9d01-0c5fa85aaafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:59.672446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:59.723291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 707/1398] 10.222.0.78 () {58 vars in 1040 bytes} [Mon Sep 19 11:17:59 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:59.733392 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4044b5-cf74-4b73-8a37-267447d98e0f None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:59.735040 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7f4044b5-cf74-4b73-8a37-267447d98e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:59.759417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:59.762931 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e218a-38dc-4014-9d01-0c5fa85aaafe None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:59.764245 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b61e218a-38dc-4014-9d01-0c5fa85aaafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:59.877204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e218a-38dc-4014-9d01-0c5fa85aaafe None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:59.878833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4044b5-cf74-4b73-8a37-267447d98e0f None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:59.953678 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:59.989520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d58982-08df-4c50-a56b-aecbdba3863a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf462b2aa3fc4e2ca58851549cf0c644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:59.990130 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d58982-08df-4c50-a56b-aecbdba3863a service swift] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:59.990399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d58982-08df-4c50-a56b-aecbdba3863a service swift] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:59.990666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d58982-08df-4c50-a56b-aecbdba3863a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:59.992415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-19d58982-08df-4c50-a56b-aecbdba3863a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:59.996345 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00f603-89a7-4f10-b2db-9976ada9e030 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:59.996956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00f603-89a7-4f10-b2db-9976ada9e030 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:59.997236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00f603-89a7-4f10-b2db-9976ada9e030 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:59.997509 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00f603-89a7-4f10-b2db-9976ada9e030 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:59.999108 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3a00f603-89a7-4f10-b2db-9976ada9e030 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:00.000124 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:00.078752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d58982-08df-4c50-a56b-aecbdba3863a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:00.080391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d58982-08df-4c50-a56b-aecbdba3863a service swift] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:00.088406 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 692/1399] 10.222.0.78 () {60 vars in 1384 bytes} [Mon Sep 19 11:17:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 11:18:00.094211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00f603-89a7-4f10-b2db-9976ada9e030 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:00.095858 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00f603-89a7-4f10-b2db-9976ada9e030 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:00.127920 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 708/1400] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:18:02.571217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663f208-e94e-4d31-b25d-7afa815db84e None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:02.571808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663f208-e94e-4d31-b25d-7afa815db84e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:02.571808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663f208-e94e-4d31-b25d-7afa815db84e None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:02.573143 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 693/1401] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:02.578677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2790d8-e648-43eb-ab07-3821493304b0 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:02.579240 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2790d8-e648-43eb-ab07-3821493304b0 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:02.579345 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2790d8-e648-43eb-ab07-3821493304b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:02.650585 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4b2790d8-e648-43eb-ab07-3821493304b0 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:02.651845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4b2790d8-e648-43eb-ab07-3821493304b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:02.693044 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:02.757825 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 709/1402] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:03.052840 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663f208-e94e-4d31-b25d-7afa815db84e None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:03.054381 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-0663f208-e94e-4d31-b25d-7afa815db84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:03.228900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663f208-e94e-4d31-b25d-7afa815db84e None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:03.233550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:03.353627 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213adf57-eac1-4c98-923f-4f12a7b7dce9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:03.354144 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213adf57-eac1-4c98-923f-4f12a7b7dce9 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:03.354417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213adf57-eac1-4c98-923f-4f12a7b7dce9 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:03.354686 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213adf57-eac1-4c98-923f-4f12a7b7dce9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:03.356317 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-213adf57-eac1-4c98-923f-4f12a7b7dce9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:03.465098 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213adf57-eac1-4c98-923f-4f12a7b7dce9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:03.466760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213adf57-eac1-4c98-923f-4f12a7b7dce9 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:03.495369 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 694/1403] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:05.992829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866e7ba-f352-4b82-bfd4-5390f518858c None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:05.993991 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866e7ba-f352-4b82-bfd4-5390f518858c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:05.994499 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866e7ba-f352-4b82-bfd4-5390f518858c None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:05.996118 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 710/1404] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:18:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:06.002425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb1937-0c52-48c0-8559-9c839a9babe6 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:06.002937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb1937-0c52-48c0-8559-9c839a9babe6 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:06.003043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb1937-0c52-48c0-8559-9c839a9babe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:06.014104 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:06.076853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-9fdb1937-0c52-48c0-8559-9c839a9babe6 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:06.078103 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-9fdb1937-0c52-48c0-8559-9c839a9babe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:06.182248 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 695/1405] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:06.499879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866e7ba-f352-4b82-bfd4-5390f518858c None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:06.501387 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c866e7ba-f352-4b82-bfd4-5390f518858c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:06.683108 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866e7ba-f352-4b82-bfd4-5390f518858c None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:06.706331 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:06.825136 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624aae3f-91a7-4169-9d1b-10047df07864 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:06.825817 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624aae3f-91a7-4169-9d1b-10047df07864 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:06.826103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624aae3f-91a7-4169-9d1b-10047df07864 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:06.826379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624aae3f-91a7-4169-9d1b-10047df07864 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:06.828088 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-624aae3f-91a7-4169-9d1b-10047df07864 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:06.933951 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624aae3f-91a7-4169-9d1b-10047df07864 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:06.950618 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624aae3f-91a7-4169-9d1b-10047df07864 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:06.959426 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 711/1406] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:09.932282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4474d35-2200-4282-b97c-3039f4aa0969 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:09.932282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4474d35-2200-4282-b97c-3039f4aa0969 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:09.932959 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4474d35-2200-4282-b97c-3039f4aa0969 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:09.933874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 696/1407] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:09.940046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce674a-91ac-4376-8ed7-8f186e84c407 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:09.940536 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce674a-91ac-4376-8ed7-8f186e84c407 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:09.940637 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce674a-91ac-4376-8ed7-8f186e84c407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:09.976979 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:10.018681 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d1ce674a-91ac-4376-8ed7-8f186e84c407 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:10.020060 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d1ce674a-91ac-4376-8ed7-8f186e84c407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:10.137142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 712/1408] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:10.429083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4474d35-2200-4282-b97c-3039f4aa0969 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:10.430612 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f4474d35-2200-4282-b97c-3039f4aa0969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:10.528429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:10.594981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4474d35-2200-4282-b97c-3039f4aa0969 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:10.720418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8673a24-b4c6-4750-8ac0-e9f8e9df7f3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:10.720945 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8673a24-b4c6-4750-8ac0-e9f8e9df7f3f service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:10.721229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8673a24-b4c6-4750-8ac0-e9f8e9df7f3f service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:10.721501 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8673a24-b4c6-4750-8ac0-e9f8e9df7f3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:10.723118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c8673a24-b4c6-4750-8ac0-e9f8e9df7f3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:10.781859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:10.825965 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8673a24-b4c6-4750-8ac0-e9f8e9df7f3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:10.827608 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8673a24-b4c6-4750-8ac0-e9f8e9df7f3f service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:10.849082 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 697/1409] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:20.135339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=54 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:20.844492 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=24 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:24.036996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b534e-f11f-4041-82f6-49a77459e24c None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:24.037570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b534e-f11f-4041-82f6-49a77459e24c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:24.037570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b534e-f11f-4041-82f6-49a77459e24c None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:24.038888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 713/1410] 10.222.0.78 () {54 vars in 918 bytes} [Mon Sep 19 11:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:24.044114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf816a-b709-4912-a7cf-01e470068e47 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:24.044591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf816a-b709-4912-a7cf-01e470068e47 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:24.044690 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf816a-b709-4912-a7cf-01e470068e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:24.121889 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-65cf816a-b709-4912-a7cf-01e470068e47 None None] MFA Rules not processed for user `0427cdd3905a422cab4fed0aac187ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:24.123116 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-65cf816a-b709-4912-a7cf-01e470068e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:24.230113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 698/1411] 10.222.0.78 () {58 vars in 1015 bytes} [Mon Sep 19 11:18:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:24.245150 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c1384-c287-404e-8a01-cc5361f07d0c None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:24.245607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c1384-c287-404e-8a01-cc5361f07d0c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:24.245884 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c1384-c287-404e-8a01-cc5361f07d0c None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:24.247262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 714/1412] 10.222.0.78 () {54 vars in 918 bytes} [Mon Sep 19 11:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:24.253108 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb5259-4a6e-4455-aaa1-9a5a90e29f65 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:24.253391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb5259-4a6e-4455-aaa1-9a5a90e29f65 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:24.253664 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb5259-4a6e-4455-aaa1-9a5a90e29f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:24.328736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:24.331347 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-46cb5259-4a6e-4455-aaa1-9a5a90e29f65 None None] MFA Rules not processed for user `aba1968529574fe5b4c5d114c874f428`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:24.332578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-46cb5259-4a6e-4455-aaa1-9a5a90e29f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:24.449703 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 699/1413] 10.222.0.78 () {58 vars in 1015 bytes} [Mon Sep 19 11:18:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:27.044567 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d40a5f-681e-47fa-8ec8-fb34b7908cb1 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:27.045178 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d40a5f-681e-47fa-8ec8-fb34b7908cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:27.045178 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d40a5f-681e-47fa-8ec8-fb34b7908cb1 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:27.046406 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:27.051813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c52aae-8011-4e5c-b813-e164f1fc029f None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:27.052282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c52aae-8011-4e5c-b813-e164f1fc029f None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:27.052394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c52aae-8011-4e5c-b813-e164f1fc029f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:27.120907 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-16c52aae-8011-4e5c-b813-e164f1fc029f None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:27.122164 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-16c52aae-8011-4e5c-b813-e164f1fc029f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:27.178162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:27.220117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:27.566399 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d40a5f-681e-47fa-8ec8-fb34b7908cb1 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:27.566399 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f6d40a5f-681e-47fa-8ec8-fb34b7908cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:27.684343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d40a5f-681e-47fa-8ec8-fb34b7908cb1 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:27.761796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:27.801466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f32fbb-288b-4887-adcc-c1bc7487b86b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:27.802009 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f32fbb-288b-4887-adcc-c1bc7487b86b service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:27.802291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f32fbb-288b-4887-adcc-c1bc7487b86b service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:27.802565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f32fbb-288b-4887-adcc-c1bc7487b86b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:27.804425 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-72f32fbb-288b-4887-adcc-c1bc7487b86b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:27.898093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f32fbb-288b-4887-adcc-c1bc7487b86b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:27.899743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f32fbb-288b-4887-adcc-c1bc7487b86b service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:27.919124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 716/1416] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:29.112800 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcc492-0119-4720-a18c-7c907d8a7a0e None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:29.113354 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcc492-0119-4720-a18c-7c907d8a7a0e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:29.113354 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcc492-0119-4720-a18c-7c907d8a7a0e None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:29.114746 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:29.121540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a238b-565d-4808-90ab-a802f2ae714e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:29.123912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a238b-565d-4808-90ab-a802f2ae714e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:29.123912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a238b-565d-4808-90ab-a802f2ae714e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:29.198969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-481a238b-565d-4808-90ab-a802f2ae714e None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:29.200291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-481a238b-565d-4808-90ab-a802f2ae714e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:29.228503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:29.297396 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:29.452519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcc492-0119-4720-a18c-7c907d8a7a0e None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:29.453972 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-0fdcc492-0119-4720-a18c-7c907d8a7a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:29.603341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcc492-0119-4720-a18c-7c907d8a7a0e None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:29.644475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:29.728967 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9aa5e-ec5b-49f9-8c3a-b33e878335e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:29.729773 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9aa5e-ec5b-49f9-8c3a-b33e878335e7 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:29.730196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9aa5e-ec5b-49f9-8c3a-b33e878335e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:29.730882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9aa5e-ec5b-49f9-8c3a-b33e878335e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:29.733395 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5ed9aa5e-ec5b-49f9-8c3a-b33e878335e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:29.839513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed9aa5e-ec5b-49f9-8c3a-b33e878335e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:29.841117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed9aa5e-ec5b-49f9-8c3a-b33e878335e7 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:29.859442 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 702/1419] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:31.008666 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4567e4-42f6-488c-a234-fd24d634e523 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:31.009144 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4567e4-42f6-488c-a234-fd24d634e523 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:31.009252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4567e4-42f6-488c-a234-fd24d634e523 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:31.011520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:18:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:31.018994 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d1c34-aa80-4980-8fe9-da6faeec0a86 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:31.019751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d1c34-aa80-4980-8fe9-da6faeec0a86 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:31.019751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d1c34-aa80-4980-8fe9-da6faeec0a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:31.074149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:31.097293 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-425d1c34-aa80-4980-8fe9-da6faeec0a86 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:31.098538 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-425d1c34-aa80-4980-8fe9-da6faeec0a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:31.312900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:31.466841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4567e4-42f6-488c-a234-fd24d634e523 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:31.468480 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0a4567e4-42f6-488c-a234-fd24d634e523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:31.613783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4567e4-42f6-488c-a234-fd24d634e523 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:31.626445 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:31.726409 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208e616-44e9-4044-a971-1177e0b4719d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:31.727016 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208e616-44e9-4044-a971-1177e0b4719d service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:31.727333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208e616-44e9-4044-a971-1177e0b4719d service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:31.727606 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208e616-44e9-4044-a971-1177e0b4719d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:31.729220 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9208e616-44e9-4044-a971-1177e0b4719d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:31.819424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208e616-44e9-4044-a971-1177e0b4719d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:31.821012 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208e616-44e9-4044-a971-1177e0b4719d service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:31.839287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 719/1422] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:32.965231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e096d3a-fff9-4c74-b903-008f0eebe959 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:32.965764 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e096d3a-fff9-4c74-b903-008f0eebe959 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:32.965764 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e096d3a-fff9-4c74-b903-008f0eebe959 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:32.967054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:32.973563 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2774a1a-f18a-46c3-896e-926d54128182 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:32.974070 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2774a1a-f18a-46c3-896e-926d54128182 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:32.974167 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2774a1a-f18a-46c3-896e-926d54128182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:32.995148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:33.053440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f2774a1a-f18a-46c3-896e-926d54128182 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:33.054674 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f2774a1a-f18a-46c3-896e-926d54128182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:33.152469 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:33.300632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e096d3a-fff9-4c74-b903-008f0eebe959 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:33.302182 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-0e096d3a-fff9-4c74-b903-008f0eebe959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:33.415176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:33.450777 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e096d3a-fff9-4c74-b903-008f0eebe959 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:33.566502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ff9c1-2a4d-447b-8cba-4d307c0b6e7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:33.567055 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ff9c1-2a4d-447b-8cba-4d307c0b6e7f service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:33.567358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ff9c1-2a4d-447b-8cba-4d307c0b6e7f service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:33.567624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ff9c1-2a4d-447b-8cba-4d307c0b6e7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:33.569216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e07ff9c1-2a4d-447b-8cba-4d307c0b6e7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:33.645617 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:33.659315 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ff9c1-2a4d-447b-8cba-4d307c0b6e7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:33.660910 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ff9c1-2a4d-447b-8cba-4d307c0b6e7f service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:33.679588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 705/1425] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:34.793623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32facf-f83d-48f1-b361-edc5033fc59e None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:34.794185 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32facf-f83d-48f1-b361-edc5033fc59e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:34.794185 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32facf-f83d-48f1-b361-edc5033fc59e None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:34.796188 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:18:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:34.802564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86beb231-1973-4849-9fbd-be78a234a2aa None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:34.803036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86beb231-1973-4849-9fbd-be78a234a2aa None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:34.803147 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86beb231-1973-4849-9fbd-be78a234a2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:34.880332 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-86beb231-1973-4849-9fbd-be78a234a2aa None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:34.881556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-86beb231-1973-4849-9fbd-be78a234a2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:34.978015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:35.130073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32facf-f83d-48f1-b361-edc5033fc59e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:35.131621 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-dd32facf-f83d-48f1-b361-edc5033fc59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:35.215548 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:35.276666 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32facf-f83d-48f1-b361-edc5033fc59e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:35.388022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa139b3-a3f8-4ddb-b1d6-a2197e6d0a63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:35.389146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa139b3-a3f8-4ddb-b1d6-a2197e6d0a63 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:35.389419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa139b3-a3f8-4ddb-b1d6-a2197e6d0a63 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:35.389689 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa139b3-a3f8-4ddb-b1d6-a2197e6d0a63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:35.391345 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4aa139b3-a3f8-4ddb-b1d6-a2197e6d0a63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:35.440487 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:35.482541 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa139b3-a3f8-4ddb-b1d6-a2197e6d0a63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:35.484156 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa139b3-a3f8-4ddb-b1d6-a2197e6d0a63 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:35.502256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 722/1428] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:36.617800 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84c6f0-131e-4722-89f6-83546961be56 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:36.618380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84c6f0-131e-4722-89f6-83546961be56 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:36.618380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84c6f0-131e-4722-89f6-83546961be56 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:36.619711 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:36.626396 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0ea24-7111-4ce2-9aa3-29d526388625 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:36.626861 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0ea24-7111-4ce2-9aa3-29d526388625 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:36.626956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0ea24-7111-4ce2-9aa3-29d526388625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:36.712267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-04c0ea24-7111-4ce2-9aa3-29d526388625 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:36.713557 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-04c0ea24-7111-4ce2-9aa3-29d526388625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:36.810116 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:36.976397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84c6f0-131e-4722-89f6-83546961be56 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:36.977824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-db84c6f0-131e-4722-89f6-83546961be56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:37.027795 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:37.125048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84c6f0-131e-4722-89f6-83546961be56 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:37.242113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46722e2-b720-4fb6-84be-385a2b3b1665 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:37.242633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46722e2-b720-4fb6-84be-385a2b3b1665 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:37.242940 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46722e2-b720-4fb6-84be-385a2b3b1665 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:37.243239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46722e2-b720-4fb6-84be-385a2b3b1665 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:37.245115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e46722e2-b720-4fb6-84be-385a2b3b1665 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:37.252768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:37.334594 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46722e2-b720-4fb6-84be-385a2b3b1665 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:37.336211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46722e2-b720-4fb6-84be-385a2b3b1665 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:37.353908 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 708/1431] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:38.490417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905071e0-c4f6-4863-80d2-599c4ed4db39 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:38.490932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905071e0-c4f6-4863-80d2-599c4ed4db39 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:38.490932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905071e0-c4f6-4863-80d2-599c4ed4db39 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:38.492291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:38.499253 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd0d44-5f09-4a80-9a12-4b8531ffbf19 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:38.499740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd0d44-5f09-4a80-9a12-4b8531ffbf19 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:38.499850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd0d44-5f09-4a80-9a12-4b8531ffbf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:38.575129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-0ccd0d44-5f09-4a80-9a12-4b8531ffbf19 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:38.576927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-0ccd0d44-5f09-4a80-9a12-4b8531ffbf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:38.643808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:38.671815 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:38.823275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905071e0-c4f6-4863-80d2-599c4ed4db39 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:38.824858 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-905071e0-c4f6-4863-80d2-599c4ed4db39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:38.843755 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:38.973513 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905071e0-c4f6-4863-80d2-599c4ed4db39 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:39.061040 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:39.087008 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713560d-520e-480b-ad1b-c2d48ea7eae8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:39.087577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713560d-520e-480b-ad1b-c2d48ea7eae8 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:39.087858 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713560d-520e-480b-ad1b-c2d48ea7eae8 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:39.088128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713560d-520e-480b-ad1b-c2d48ea7eae8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:39.089763 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3713560d-520e-480b-ad1b-c2d48ea7eae8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:39.180993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713560d-520e-480b-ad1b-c2d48ea7eae8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:39.182611 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713560d-520e-480b-ad1b-c2d48ea7eae8 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:39.201742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 725/1434] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:40.380773 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63afc7-eeca-466f-9301-1af14ced7128 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:40.381305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63afc7-eeca-466f-9301-1af14ced7128 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:40.381305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63afc7-eeca-466f-9301-1af14ced7128 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:40.382580 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:40.389397 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfff7b-4670-4be8-8fea-9eda26ebbab1 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:40.389860 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfff7b-4670-4be8-8fea-9eda26ebbab1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:40.389955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfff7b-4670-4be8-8fea-9eda26ebbab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:40.466096 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-cbbfff7b-4670-4be8-8fea-9eda26ebbab1 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:40.468080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-cbbfff7b-4670-4be8-8fea-9eda26ebbab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:40.509245 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:40.563105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:40.714875 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63afc7-eeca-466f-9301-1af14ced7128 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:40.717347 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-6a63afc7-eeca-466f-9301-1af14ced7128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:40.866834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63afc7-eeca-466f-9301-1af14ced7128 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:40.922391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:40.998547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7221ae7-08c3-4534-8e48-9be613a23083 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:40.999446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7221ae7-08c3-4534-8e48-9be613a23083 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:40.999867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7221ae7-08c3-4534-8e48-9be613a23083 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:41.000274 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7221ae7-08c3-4534-8e48-9be613a23083 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:41.002766 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a7221ae7-08c3-4534-8e48-9be613a23083 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:41.103875 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7221ae7-08c3-4534-8e48-9be613a23083 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:41.105877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7221ae7-08c3-4534-8e48-9be613a23083 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:41.125369 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 711/1437] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:42.276750 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1e34f-6797-4464-bdfe-3279f50d58e5 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:42.277286 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1e34f-6797-4464-bdfe-3279f50d58e5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:42.277286 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1e34f-6797-4464-bdfe-3279f50d58e5 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:42.278689 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:42.285212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26673ff5-962b-4610-ae38-91bda88f1dab None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:42.285682 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26673ff5-962b-4610-ae38-91bda88f1dab None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:42.285780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26673ff5-962b-4610-ae38-91bda88f1dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:42.360426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-26673ff5-962b-4610-ae38-91bda88f1dab None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:42.361631 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-26673ff5-962b-4610-ae38-91bda88f1dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:42.363660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:42.459028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:42.609645 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1e34f-6797-4464-bdfe-3279f50d58e5 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:42.611228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c2e1e34f-6797-4464-bdfe-3279f50d58e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:42.761467 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1e34f-6797-4464-bdfe-3279f50d58e5 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:42.799680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:42.887643 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d45a1-f505-4b01-8a49-23fcd2e39fa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:42.888261 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d45a1-f505-4b01-8a49-23fcd2e39fa1 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:42.888561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d45a1-f505-4b01-8a49-23fcd2e39fa1 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:42.888834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d45a1-f505-4b01-8a49-23fcd2e39fa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:42.890446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-651d45a1-f505-4b01-8a49-23fcd2e39fa1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:42.983955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d45a1-f505-4b01-8a49-23fcd2e39fa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:42.985539 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d45a1-f505-4b01-8a49-23fcd2e39fa1 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:43.003917 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 728/1440] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:45.072493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397baf7-bdb1-46f6-8b6b-e04edf6494d1 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:45.073088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397baf7-bdb1-46f6-8b6b-e04edf6494d1 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:45.073088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397baf7-bdb1-46f6-8b6b-e04edf6494d1 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:45.074358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:45.080196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f2bf6-1ff9-49e2-ac1a-79237977f87d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:45.080635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f2bf6-1ff9-49e2-ac1a-79237977f87d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:45.080735 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f2bf6-1ff9-49e2-ac1a-79237977f87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:45.122712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:45.148066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8f3f2bf6-1ff9-49e2-ac1a-79237977f87d None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:45.149290 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8f3f2bf6-1ff9-49e2-ac1a-79237977f87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:45.245202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:45.532151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397baf7-bdb1-46f6-8b6b-e04edf6494d1 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:45.533696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-8397baf7-bdb1-46f6-8b6b-e04edf6494d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:45.636398 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397baf7-bdb1-46f6-8b6b-e04edf6494d1 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:45.687869 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:45.736240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef30b9-f1bb-4786-87e1-0f91618edd71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:45.736841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef30b9-f1bb-4786-87e1-0f91618edd71 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:45.737123 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef30b9-f1bb-4786-87e1-0f91618edd71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:45.737397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef30b9-f1bb-4786-87e1-0f91618edd71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:45.739020 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-bfef30b9-f1bb-4786-87e1-0f91618edd71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:45.849662 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef30b9-f1bb-4786-87e1-0f91618edd71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:45.850811 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef30b9-f1bb-4786-87e1-0f91618edd71 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:45.872114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 714/1443] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:48.290079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15982d1-c88d-4bdc-967e-b8fdc71a4a60 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:48.290650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15982d1-c88d-4bdc-967e-b8fdc71a4a60 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:48.290650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15982d1-c88d-4bdc-967e-b8fdc71a4a60 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:48.292125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:48.297498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bde2c4-3d64-4d9f-984a-fcbd640db403 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:48.298023 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bde2c4-3d64-4d9f-984a-fcbd640db403 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:48.298118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bde2c4-3d64-4d9f-984a-fcbd640db403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:48.365353 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-27bde2c4-3d64-4d9f-984a-fcbd640db403 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:48.366491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-27bde2c4-3d64-4d9f-984a-fcbd640db403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:48.380441 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:48.466835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:48.754519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15982d1-c88d-4bdc-967e-b8fdc71a4a60 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:48.756142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d15982d1-c88d-4bdc-967e-b8fdc71a4a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:48.864788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15982d1-c88d-4bdc-967e-b8fdc71a4a60 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:48.878621 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:48.944070 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b56e01-9963-49d7-9243-434491b389e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:48.944616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b56e01-9963-49d7-9243-434491b389e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:48.944898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b56e01-9963-49d7-9243-434491b389e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:48.945173 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b56e01-9963-49d7-9243-434491b389e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:48.946836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-98b56e01-9963-49d7-9243-434491b389e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:49.060192 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b56e01-9963-49d7-9243-434491b389e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:49.061661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b56e01-9963-49d7-9243-434491b389e0 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:49.079469 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 731/1446] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:51.797560 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84a7b3-cbdb-4eb6-af3c-7df8da553e56 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:51.798155 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84a7b3-cbdb-4eb6-af3c-7df8da553e56 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:51.798155 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84a7b3-cbdb-4eb6-af3c-7df8da553e56 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:51.799649 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:51.805217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6b5d6-a088-4e11-97ec-2879df8c8604 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:51.805695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6b5d6-a088-4e11-97ec-2879df8c8604 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:51.805795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6b5d6-a088-4e11-97ec-2879df8c8604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:51.867301 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:51.888311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b1d6b5d6-a088-4e11-97ec-2879df8c8604 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:51.889967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b1d6b5d6-a088-4e11-97ec-2879df8c8604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:51.991581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:52.284753 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84a7b3-cbdb-4eb6-af3c-7df8da553e56 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:52.286297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-fc84a7b3-cbdb-4eb6-af3c-7df8da553e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:52.396365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84a7b3-cbdb-4eb6-af3c-7df8da553e56 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:52.437216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:52.470889 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab74818-7c3c-49a7-8772-46c37c76487b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:52.471553 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab74818-7c3c-49a7-8772-46c37c76487b service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:52.471829 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab74818-7c3c-49a7-8772-46c37c76487b service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:52.472098 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab74818-7c3c-49a7-8772-46c37c76487b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:52.473704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2ab74818-7c3c-49a7-8772-46c37c76487b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:52.582155 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab74818-7c3c-49a7-8772-46c37c76487b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:52.583628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab74818-7c3c-49a7-8772-46c37c76487b service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:52.602630 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 717/1449] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:18:54.915405 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f5da7-ed00-49ac-9956-11ada69da2ea None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:54.915993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f5da7-ed00-49ac-9956-11ada69da2ea None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:54.915993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f5da7-ed00-49ac-9956-11ada69da2ea None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:54.917366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:54.923062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb7f8b-dc21-4bcb-83dc-5e6598424328 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:54.923490 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb7f8b-dc21-4bcb-83dc-5e6598424328 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:54.923637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb7f8b-dc21-4bcb-83dc-5e6598424328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:54.989276 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-97cb7f8b-dc21-4bcb-83dc-5e6598424328 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:54.990493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-97cb7f8b-dc21-4bcb-83dc-5e6598424328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:55.011075 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:55.085226 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:55.366923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f5da7-ed00-49ac-9956-11ada69da2ea None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:55.368515 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2f1f5da7-ed00-49ac-9956-11ada69da2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:55.473864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f5da7-ed00-49ac-9956-11ada69da2ea None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:55.496451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:55.549728 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59398dc9-f8e0-4e41-a41b-42cc0735077a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:55.550336 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59398dc9-f8e0-4e41-a41b-42cc0735077a service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:55.550617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59398dc9-f8e0-4e41-a41b-42cc0735077a service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:55.550889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59398dc9-f8e0-4e41-a41b-42cc0735077a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:55.552535 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-59398dc9-f8e0-4e41-a41b-42cc0735077a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:55.663975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59398dc9-f8e0-4e41-a41b-42cc0735077a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:55.665430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59398dc9-f8e0-4e41-a41b-42cc0735077a service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:55.683410 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 734/1452] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:57.953796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ed12d-7067-4543-9a16-d3b0ff2fab05 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:57.954341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ed12d-7067-4543-9a16-d3b0ff2fab05 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:57.954341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ed12d-7067-4543-9a16-d3b0ff2fab05 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:57.955648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:57.961084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d60f49-3da4-49c8-9137-8c9255996b72 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:57.961523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d60f49-3da4-49c8-9137-8c9255996b72 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:57.961632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d60f49-3da4-49c8-9137-8c9255996b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:58.028114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:58.031565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-42d60f49-3da4-49c8-9137-8c9255996b72 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:58.032826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-42d60f49-3da4-49c8-9137-8c9255996b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:58.130458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:58.419077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ed12d-7067-4543-9a16-d3b0ff2fab05 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:58.421198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7e6ed12d-7067-4543-9a16-d3b0ff2fab05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:58.524832 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ed12d-7067-4543-9a16-d3b0ff2fab05 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:58.574104 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:58.598898 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a541a-a604-44d7-8e73-f5189218485d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:58.599532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a541a-a604-44d7-8e73-f5189218485d service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:58.599811 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a541a-a604-44d7-8e73-f5189218485d service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:58.600103 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a541a-a604-44d7-8e73-f5189218485d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:58.601710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1e1a541a-a604-44d7-8e73-f5189218485d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:58.711058 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a541a-a604-44d7-8e73-f5189218485d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:58.712521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a541a-a604-44d7-8e73-f5189218485d service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:58.731597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 720/1455] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:19:01.067435 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99910f1-c56a-4dc5-b573-4ffb8d871b3d None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:01.068176 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99910f1-c56a-4dc5-b573-4ffb8d871b3d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:01.068176 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99910f1-c56a-4dc5-b573-4ffb8d871b3d None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:01.069333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:19:01.075021 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e4d6e-6014-43dc-b676-5f9bc617cc4a None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:01.075489 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e4d6e-6014-43dc-b676-5f9bc617cc4a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:01.075617 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e4d6e-6014-43dc-b676-5f9bc617cc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:01.146332 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-d43e4d6e-6014-43dc-b676-5f9bc617cc4a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:01.147595 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d43e4d6e-6014-43dc-b676-5f9bc617cc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:01.178157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:01.246526 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:19:01.530275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99910f1-c56a-4dc5-b573-4ffb8d871b3d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:01.531885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b99910f1-c56a-4dc5-b573-4ffb8d871b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:01.640417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99910f1-c56a-4dc5-b573-4ffb8d871b3d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:01.679868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:01.728870 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26835523-5cbe-4f90-bfb7-92ba93b09c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:01.729730 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26835523-5cbe-4f90-bfb7-92ba93b09c15 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:01.730025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26835523-5cbe-4f90-bfb7-92ba93b09c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:01.730298 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26835523-5cbe-4f90-bfb7-92ba93b09c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:01.732440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-26835523-5cbe-4f90-bfb7-92ba93b09c15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:01.842969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26835523-5cbe-4f90-bfb7-92ba93b09c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:01.844438 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26835523-5cbe-4f90-bfb7-92ba93b09c15 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:01.863566 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 737/1458] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:19:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:19:04.080089 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a638540-183a-4463-9a92-eb6c6a8c6032 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:04.080909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a638540-183a-4463-9a92-eb6c6a8c6032 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:04.081094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a638540-183a-4463-9a92-eb6c6a8c6032 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:04.082799 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:19:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:19:04.089003 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e154a1-b306-4b10-aaeb-c0e2e4e0cadf None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:04.089538 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e154a1-b306-4b10-aaeb-c0e2e4e0cadf None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:04.089852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e154a1-b306-4b10-aaeb-c0e2e4e0cadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:04.167958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-71e154a1-b306-4b10-aaeb-c0e2e4e0cadf None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:04.169201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-71e154a1-b306-4b10-aaeb-c0e2e4e0cadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:04.178671 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:04.268822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:19:04.549517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a638540-183a-4463-9a92-eb6c6a8c6032 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:04.551091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-0a638540-183a-4463-9a92-eb6c6a8c6032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:04.654946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a638540-183a-4463-9a92-eb6c6a8c6032 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:04.710488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:04.729197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074127f-c7eb-4ff3-864e-f8c1dc73b0a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:04.731211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074127f-c7eb-4ff3-864e-f8c1dc73b0a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:04.731211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074127f-c7eb-4ff3-864e-f8c1dc73b0a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:04.731211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074127f-c7eb-4ff3-864e-f8c1dc73b0a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:04.732192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-0074127f-c7eb-4ff3-864e-f8c1dc73b0a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:04.842319 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074127f-c7eb-4ff3-864e-f8c1dc73b0a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:04.843888 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074127f-c7eb-4ff3-864e-f8c1dc73b0a1 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:04.862925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 723/1461] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:19:07.102047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959cb9c-5fd0-460c-ad7d-d773b3d27f0d None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:07.102614 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959cb9c-5fd0-460c-ad7d-d773b3d27f0d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:07.102614 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959cb9c-5fd0-460c-ad7d-d773b3d27f0d None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:07.103979 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:19:07.109941 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3b1fd-91b9-4e12-bad8-1894013aeeba None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:07.110425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3b1fd-91b9-4e12-bad8-1894013aeeba None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:07.110510 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3b1fd-91b9-4e12-bad8-1894013aeeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:07.176880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-bac3b1fd-91b9-4e12-bad8-1894013aeeba None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:07.178133 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-bac3b1fd-91b9-4e12-bad8-1894013aeeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:07.217467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:07.274641 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:19:07.564897 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959cb9c-5fd0-460c-ad7d-d773b3d27f0d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:07.566486 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0959cb9c-5fd0-460c-ad7d-d773b3d27f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:07.671042 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959cb9c-5fd0-460c-ad7d-d773b3d27f0d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:07.709051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:07.744848 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2c0c3-e5db-439e-aaba-f6a60c9f2032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:07.745452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2c0c3-e5db-439e-aaba-f6a60c9f2032 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:07.745731 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2c0c3-e5db-439e-aaba-f6a60c9f2032 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:07.746000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2c0c3-e5db-439e-aaba-f6a60c9f2032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:07.748167 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-94c2c0c3-e5db-439e-aaba-f6a60c9f2032 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:07.855215 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2c0c3-e5db-439e-aaba-f6a60c9f2032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:07.856590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2c0c3-e5db-439e-aaba-f6a60c9f2032 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:07.876200 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 740/1464] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:19:10.098696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f628a-d26c-45da-8131-38e5573aa444 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:10.099303 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f628a-d26c-45da-8131-38e5573aa444 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:10.099303 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f628a-d26c-45da-8131-38e5573aa444 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:10.100638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:19:10.106301 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44405ede-eeab-4de4-b478-ee9f900c7495 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:10.107295 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44405ede-eeab-4de4-b478-ee9f900c7495 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:10.107463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44405ede-eeab-4de4-b478-ee9f900c7495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:10.172529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-44405ede-eeab-4de4-b478-ee9f900c7495 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:10.173754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-44405ede-eeab-4de4-b478-ee9f900c7495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:10.192677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:10.270924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:19:10.554018 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f628a-d26c-45da-8131-38e5573aa444 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:10.555629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-068f628a-d26c-45da-8131-38e5573aa444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:10.660517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f628a-d26c-45da-8131-38e5573aa444 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:10.726969 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:10.734254 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f296b-68e8-49f9-af4a-050617cbfd86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:10.740890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f296b-68e8-49f9-af4a-050617cbfd86 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:10.740890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f296b-68e8-49f9-af4a-050617cbfd86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:10.740890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f296b-68e8-49f9-af4a-050617cbfd86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:10.740890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c92f296b-68e8-49f9-af4a-050617cbfd86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:10.846534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f296b-68e8-49f9-af4a-050617cbfd86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:10.847934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f296b-68e8-49f9-af4a-050617cbfd86 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:10.866555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 726/1467] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:19:13.049470 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b638b5-f5dd-496a-b7fa-f5a6df65dd3d None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:13.050047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b638b5-f5dd-496a-b7fa-f5a6df65dd3d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:13.050047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b638b5-f5dd-496a-b7fa-f5a6df65dd3d None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:13.051523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:19:13.057079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4931538-926d-4d1b-b18e-5b27b908b428 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:13.057528 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4931538-926d-4d1b-b18e-5b27b908b428 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:13.057635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4931538-926d-4d1b-b18e-5b27b908b428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:13.133726 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-f4931538-926d-4d1b-b18e-5b27b908b428 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:13.134962 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f4931538-926d-4d1b-b18e-5b27b908b428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:13.191334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:13.255257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:19:13.530228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b638b5-f5dd-496a-b7fa-f5a6df65dd3d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:13.531930 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-14b638b5-f5dd-496a-b7fa-f5a6df65dd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:13.637284 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b638b5-f5dd-496a-b7fa-f5a6df65dd3d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:13.687284 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:13.713266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcbdc3-0ab4-4dc1-9801-56af619158e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:13.713878 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcbdc3-0ab4-4dc1-9801-56af619158e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:13.714153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcbdc3-0ab4-4dc1-9801-56af619158e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:13.714424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcbdc3-0ab4-4dc1-9801-56af619158e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:13.716089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-75bcbdc3-0ab4-4dc1-9801-56af619158e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:13.829666 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcbdc3-0ab4-4dc1-9801-56af619158e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:13.829666 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcbdc3-0ab4-4dc1-9801-56af619158e0 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:13.845907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 743/1470] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:19:23.246475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=22 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:23.843692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=66 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:34.049641 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c250b-7293-48bc-ad75-7d5512df95ec None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:34.050217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c250b-7293-48bc-ad75-7d5512df95ec None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:34.050217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c250b-7293-48bc-ad75-7d5512df95ec None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:34.051588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 728/1471] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:19:34.057040 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3b7f1-d26d-4179-a218-19ea5a494e19 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:34.057447 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3b7f1-d26d-4179-a218-19ea5a494e19 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:34.057594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3b7f1-d26d-4179-a218-19ea5a494e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:34.128417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4ce3b7f1-d26d-4179-a218-19ea5a494e19 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:34.129685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4ce3b7f1-d26d-4179-a218-19ea5a494e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:34.225428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 744/1472] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:19:34.526254 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f722276-689e-4c35-9377-71618396f07c None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:34.526546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f722276-689e-4c35-9377-71618396f07c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:34.526819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f722276-689e-4c35-9377-71618396f07c None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:34.528820 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 729/1473] 10.222.0.78 () {54 vars in 962 bytes} [Mon Sep 19 11:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:19:34.533936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e133fed-179c-4019-833d-d636ff1e7b12 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:34.534228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e133fed-179c-4019-833d-d636ff1e7b12 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:34.534505 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e133fed-179c-4019-833d-d636ff1e7b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:34.611037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-6e133fed-179c-4019-833d-d636ff1e7b12 None None] MFA Rules not processed for user `558e3b6dd9754c57ad028aa431a68b80`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:34.611894 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6e133fed-179c-4019-833d-d636ff1e7b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:34.631982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:34.687460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 745/1474] 10.222.0.78 () {58 vars in 1059 bytes} [Mon Sep 19 11:19:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:19:34.695396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f722276-689e-4c35-9377-71618396f07c None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:34.696956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4f722276-689e-4c35-9377-71618396f07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:34.800335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f722276-689e-4c35-9377-71618396f07c None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:34.874992 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54975fc1-ee15-4367-b769-115ed73e99e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558e3b6dd9754c57ad028aa431a68b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:34.875661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54975fc1-ee15-4367-b769-115ed73e99e7 service glance] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:34.875945 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54975fc1-ee15-4367-b769-115ed73e99e7 service glance] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:34.876217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54975fc1-ee15-4367-b769-115ed73e99e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:34.877835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-54975fc1-ee15-4367-b769-115ed73e99e7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:34.931690 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:34.985607 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54975fc1-ee15-4367-b769-115ed73e99e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:34.987114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54975fc1-ee15-4367-b769-115ed73e99e7 service glance] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:35.007095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 730/1475] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:19:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:19:37.270308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc916f-10ea-4e44-8207-0fe1fe34fb5c None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:37.270877 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc916f-10ea-4e44-8207-0fe1fe34fb5c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:37.270877 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc916f-10ea-4e44-8207-0fe1fe34fb5c None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:37.272276 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 746/1476] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:19:37.277759 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232aad5-d685-4ad2-87e6-71f0a3dba8ef None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:37.278205 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232aad5-d685-4ad2-87e6-71f0a3dba8ef None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:37.278378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232aad5-d685-4ad2-87e6-71f0a3dba8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:37.344207 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-5232aad5-d685-4ad2-87e6-71f0a3dba8ef None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:37.345406 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5232aad5-d685-4ad2-87e6-71f0a3dba8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:37.447513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 731/1477] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:19:37.721008 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc916f-10ea-4e44-8207-0fe1fe34fb5c None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:37.722497 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-41bc916f-10ea-4e44-8207-0fe1fe34fb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:37.849377 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc916f-10ea-4e44-8207-0fe1fe34fb5c None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:37.900441 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:37.946382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4e190-6911-47b6-aded-690cd69299f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:37.947415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4e190-6911-47b6-aded-690cd69299f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:37.947883 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4e190-6911-47b6-aded-690cd69299f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:37.948201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4e190-6911-47b6-aded-690cd69299f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:37.949179 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4e190-6911-47b6-aded-690cd69299f7 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:37.951598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: WARNING py.warnings [None req-7aa4e190-6911-47b6-aded-690cd69299f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:19:37.951598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: warnings.warn(msg) Sep 19 11:19:37.951598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: Sep 19 11:19:37.952705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4e190-6911-47b6-aded-690cd69299f7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:37.981684 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 747/1478] 10.222.0.78 () {56 vars in 1108 bytes} [Mon Sep 19 11:19:37 2022] GET /identity/v3/projects => generated 2553 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:19:40.137462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb82b0d-578a-4f42-9747-111bb7453144 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:40.138011 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb82b0d-578a-4f42-9747-111bb7453144 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:40.138011 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb82b0d-578a-4f42-9747-111bb7453144 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:40.139408 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 732/1479] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:19:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:19:40.145024 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285639d-6e81-47da-8bb9-74e3e3b287f9 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:40.145479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285639d-6e81-47da-8bb9-74e3e3b287f9 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:40.145613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285639d-6e81-47da-8bb9-74e3e3b287f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:40.212638 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f285639d-6e81-47da-8bb9-74e3e3b287f9 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:40.213933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f285639d-6e81-47da-8bb9-74e3e3b287f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:40.310013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 748/1480] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:19:40.612712 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb82b0d-578a-4f42-9747-111bb7453144 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:40.614247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7fb82b0d-578a-4f42-9747-111bb7453144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:40.624004 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:40.760166 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb82b0d-578a-4f42-9747-111bb7453144 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:40.838824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:40.872787 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f3788-020b-422f-8dc3-f5a8b226bc79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:40.873319 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f3788-020b-422f-8dc3-f5a8b226bc79 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:40.873599 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f3788-020b-422f-8dc3-f5a8b226bc79 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:40.873871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f3788-020b-422f-8dc3-f5a8b226bc79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:40.875509 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5a3f3788-020b-422f-8dc3-f5a8b226bc79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:40.965670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f3788-020b-422f-8dc3-f5a8b226bc79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:40.967297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f3788-020b-422f-8dc3-f5a8b226bc79 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:40.985472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 733/1481] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:19:43.356422 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb347af2-3d08-4ade-91da-8e1f1fc30828 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:43.357003 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb347af2-3d08-4ade-91da-8e1f1fc30828 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:43.357003 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb347af2-3d08-4ade-91da-8e1f1fc30828 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:43.358323 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 749/1482] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:19:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:19:43.363880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483da3e-b0eb-4a10-b7f8-57fc78360187 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:43.364380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483da3e-b0eb-4a10-b7f8-57fc78360187 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:43.364487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483da3e-b0eb-4a10-b7f8-57fc78360187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:43.439316 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-c483da3e-b0eb-4a10-b7f8-57fc78360187 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:43.440464 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c483da3e-b0eb-4a10-b7f8-57fc78360187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:43.487346 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:43.541458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 734/1483] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:19:43.836764 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb347af2-3d08-4ade-91da-8e1f1fc30828 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:43.838313 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-cb347af2-3d08-4ade-91da-8e1f1fc30828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:43.864306 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:43.943834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb347af2-3d08-4ade-91da-8e1f1fc30828 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:44.019254 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028b0be-8499-4cca-8dfa-fd0e0e496c09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:44.019873 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028b0be-8499-4cca-8dfa-fd0e0e496c09 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:44.020155 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028b0be-8499-4cca-8dfa-fd0e0e496c09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:44.020430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028b0be-8499-4cca-8dfa-fd0e0e496c09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:44.022045 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d028b0be-8499-4cca-8dfa-fd0e0e496c09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:44.098283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:44.133671 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028b0be-8499-4cca-8dfa-fd0e0e496c09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:44.135120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028b0be-8499-4cca-8dfa-fd0e0e496c09 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:44.154792 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 750/1484] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:19:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:19:46.365537 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a8695-126a-47cf-8bc9-a3f2d648c487 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:46.366105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a8695-126a-47cf-8bc9-a3f2d648c487 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:46.366105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a8695-126a-47cf-8bc9-a3f2d648c487 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:46.367386 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 735/1485] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:19:46.372843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a6ad6-e3a3-46b3-b430-27fc02567b83 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:46.373287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a6ad6-e3a3-46b3-b430-27fc02567b83 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:46.373434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a6ad6-e3a3-46b3-b430-27fc02567b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:46.440556 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-412a6ad6-e3a3-46b3-b430-27fc02567b83 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:46.441792 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-412a6ad6-e3a3-46b3-b430-27fc02567b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:46.564720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 751/1486] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:19:46.859664 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a8695-126a-47cf-8bc9-a3f2d648c487 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:46.861247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-119a8695-126a-47cf-8bc9-a3f2d648c487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:46.966047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a8695-126a-47cf-8bc9-a3f2d648c487 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:47.037889 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:47.039349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7c6e7-d747-4286-84f7-daf693356fd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:47.039989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7c6e7-d747-4286-84f7-daf693356fd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:47.040270 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7c6e7-d747-4286-84f7-daf693356fd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:47.040536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7c6e7-d747-4286-84f7-daf693356fd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:47.042132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-51b7c6e7-d747-4286-84f7-daf693356fd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:47.153096 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7c6e7-d747-4286-84f7-daf693356fd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:47.154528 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7c6e7-d747-4286-84f7-daf693356fd0 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:47.172993 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 736/1487] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:19:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:19:49.464331 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b0242-6209-4907-8a02-d18f23e99219 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:49.465390 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b0242-6209-4907-8a02-d18f23e99219 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:49.465497 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b0242-6209-4907-8a02-d18f23e99219 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:49.466816 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 752/1488] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:19:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:19:49.472251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2a5b5-54a4-4818-ab36-9ffc839f8226 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:49.472673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2a5b5-54a4-4818-ab36-9ffc839f8226 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:49.472782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2a5b5-54a4-4818-ab36-9ffc839f8226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:49.544304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-f9d2a5b5-54a4-4818-ab36-9ffc839f8226 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:49.545512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f9d2a5b5-54a4-4818-ab36-9ffc839f8226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:49.597524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:49.644537 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 737/1489] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:19:49.937221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b0242-6209-4907-8a02-d18f23e99219 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:49.938887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-139b0242-6209-4907-8a02-d18f23e99219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:49.975799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:50.045224 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b0242-6209-4907-8a02-d18f23e99219 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:50.119306 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d411ad3-ddd1-4925-a44d-f1c943d6676e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:50.128762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d411ad3-ddd1-4925-a44d-f1c943d6676e service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:50.128762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d411ad3-ddd1-4925-a44d-f1c943d6676e service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:50.128762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d411ad3-ddd1-4925-a44d-f1c943d6676e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:50.128762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9d411ad3-ddd1-4925-a44d-f1c943d6676e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:50.202675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:50.232044 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d411ad3-ddd1-4925-a44d-f1c943d6676e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:50.233536 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d411ad3-ddd1-4925-a44d-f1c943d6676e service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:50.252213 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 753/1490] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:19:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:19:52.749495 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49371fa1-1048-40d5-9e4e-3a9b691b7603 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:52.750035 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49371fa1-1048-40d5-9e4e-3a9b691b7603 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:52.750035 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49371fa1-1048-40d5-9e4e-3a9b691b7603 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:52.751437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 738/1491] 10.222.0.78 () {54 vars in 927 bytes} [Mon Sep 19 11:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:19:52.756862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f8588-ac87-4c26-98bc-117d79bb263c None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:52.757283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f8588-ac87-4c26-98bc-117d79bb263c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:52.757446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f8588-ac87-4c26-98bc-117d79bb263c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:52.822712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-478f8588-ac87-4c26-98bc-117d79bb263c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:52.824106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-478f8588-ac87-4c26-98bc-117d79bb263c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:52.921304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 754/1492] 10.222.0.78 () {58 vars in 1024 bytes} [Mon Sep 19 11:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:19:53.211394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49371fa1-1048-40d5-9e4e-3a9b691b7603 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:53.212936 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-49371fa1-1048-40d5-9e4e-3a9b691b7603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:53.321059 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49371fa1-1048-40d5-9e4e-3a9b691b7603 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:53.393305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4748abd-9268-44e9-a75a-35d6c91c001c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:53.393900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4748abd-9268-44e9-a75a-35d6c91c001c service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:53.394173 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4748abd-9268-44e9-a75a-35d6c91c001c service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:53.394444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4748abd-9268-44e9-a75a-35d6c91c001c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:53.396062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c4748abd-9268-44e9-a75a-35d6c91c001c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:53.404698 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:53.506760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4748abd-9268-44e9-a75a-35d6c91c001c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:53.508231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4748abd-9268-44e9-a75a-35d6c91c001c service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:53.526245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 739/1493] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:19:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:20:02.919575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=82 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:20:03.524192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=50 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:21:16.594460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef996b-0395-41e2-8fa3-10d4f490a84b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:16.595138 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef996b-0395-41e2-8fa3-10d4f490a84b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:16.595138 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef996b-0395-41e2-8fa3-10d4f490a84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:16.671438 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-aeef996b-0395-41e2-8fa3-10d4f490a84b None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:21:16.672675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-aeef996b-0395-41e2-8fa3-10d4f490a84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:16.771747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 755/1494] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:21:16.779889 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2261b8f-d15d-4eb4-bb32-344c0d2201cd None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:16.780419 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2261b8f-d15d-4eb4-bb32-344c0d2201cd None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:16.780419 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2261b8f-d15d-4eb4-bb32-344c0d2201cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:16.856525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-e2261b8f-d15d-4eb4-bb32-344c0d2201cd None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:21:16.857646 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e2261b8f-d15d-4eb4-bb32-344c0d2201cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:16.956022 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 740/1495] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:21:16.960804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef996b-0395-41e2-8fa3-10d4f490a84b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:16.962015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-aeef996b-0395-41e2-8fa3-10d4f490a84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:16.974024 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef996b-0395-41e2-8fa3-10d4f490a84b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:17.036331 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:21:17.071868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ec9cd-a9dc-402a-8623-6294cb0e7321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:17.072476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ec9cd-a9dc-402a-8623-6294cb0e7321 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:17.072756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ec9cd-a9dc-402a-8623-6294cb0e7321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:17.073049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ec9cd-a9dc-402a-8623-6294cb0e7321 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:17.073669 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ec9cd-a9dc-402a-8623-6294cb0e7321 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:21:17.073971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ec9cd-a9dc-402a-8623-6294cb0e7321 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:17.074984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: WARNING py.warnings [None req-b44ec9cd-a9dc-402a-8623-6294cb0e7321 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:21:17.074984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: warnings.warn(msg) Sep 19 11:21:17.074984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: Sep 19 11:21:17.075905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ec9cd-a9dc-402a-8623-6294cb0e7321 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:17.087033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 756/1496] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:21:17.095733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2261b8f-d15d-4eb4-bb32-344c0d2201cd None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:17.105679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2261b8f-d15d-4eb4-bb32-344c0d2201cd None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:17.169057 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:21:17.203260 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb6cf2-a052-4889-8b27-ae7e73cd806b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:17.203871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb6cf2-a052-4889-8b27-ae7e73cd806b admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:17.204146 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb6cf2-a052-4889-8b27-ae7e73cd806b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:17.204415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb6cf2-a052-4889-8b27-ae7e73cd806b admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:17.205044 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb6cf2-a052-4889-8b27-ae7e73cd806b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:17.208665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: WARNING py.warnings [None req-edeb6cf2-a052-4889-8b27-ae7e73cd806b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:21:17.208665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: warnings.warn(msg) Sep 19 11:21:17.208665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: Sep 19 11:21:17.209879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb6cf2-a052-4889-8b27-ae7e73cd806b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:17.453578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 741/1497] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:21:17 2022] POST /identity/v3/projects => generated 379 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:21:17.457901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ec9cd-a9dc-402a-8623-6294cb0e7321 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:17.459491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b44ec9cd-a9dc-402a-8623-6294cb0e7321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:17.494702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ec9cd-a9dc-402a-8623-6294cb0e7321 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:17.591781 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0680e97-7577-4f38-9152-a6fe9470fb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:17.592395 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0680e97-7577-4f38-9152-a6fe9470fb6d admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:17.592679 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0680e97-7577-4f38-9152-a6fe9470fb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:17.592953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0680e97-7577-4f38-9152-a6fe9470fb6d admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:17.593595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0680e97-7577-4f38-9152-a6fe9470fb6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:17.594805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: WARNING py.warnings [None req-c0680e97-7577-4f38-9152-a6fe9470fb6d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:21:17.594805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: warnings.warn(msg) Sep 19 11:21:17.594805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: Sep 19 11:21:17.596060 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0680e97-7577-4f38-9152-a6fe9470fb6d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:17.682027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 757/1498] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:21:17 2022] POST /identity/v3/users => generated 290 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:21:17.686197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb6cf2-a052-4889-8b27-ae7e73cd806b admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:17.687772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-edeb6cf2-a052-4889-8b27-ae7e73cd806b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:17.749624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb6cf2-a052-4889-8b27-ae7e73cd806b admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:17.832311 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:21:17.842872 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcfa75-8d83-42ae-915f-5bc56e31a1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:17.857413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcfa75-8d83-42ae-915f-5bc56e31a1fd admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:17.857413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcfa75-8d83-42ae-915f-5bc56e31a1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:17.857413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcfa75-8d83-42ae-915f-5bc56e31a1fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:17.857413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcfa75-8d83-42ae-915f-5bc56e31a1fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:17.857413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: WARNING py.warnings [None req-59dcfa75-8d83-42ae-915f-5bc56e31a1fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:21:17.857413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: warnings.warn(msg) Sep 19 11:21:17.857413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: Sep 19 11:21:17.858103 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcfa75-8d83-42ae-915f-5bc56e31a1fd admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:17.861203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 742/1499] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:21:17.865431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0680e97-7577-4f38-9152-a6fe9470fb6d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:17.867034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c0680e97-7577-4f38-9152-a6fe9470fb6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:17.878742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0680e97-7577-4f38-9152-a6fe9470fb6d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:17.905383 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:21:17.975198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00390b-6a0b-4d98-a82c-f60469f03ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:17.975878 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00390b-6a0b-4d98-a82c-f60469f03ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:17.976154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00390b-6a0b-4d98-a82c-f60469f03ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:17.976465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00390b-6a0b-4d98-a82c-f60469f03ce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:17.977108 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00390b-6a0b-4d98-a82c-f60469f03ce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:17.978098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: WARNING py.warnings [None req-0c00390b-6a0b-4d98-a82c-f60469f03ce7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:21:17.978098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: warnings.warn(msg) Sep 19 11:21:17.978098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: Sep 19 11:21:17.978982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00390b-6a0b-4d98-a82c-f60469f03ce7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:17.992459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 758/1500] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:21:17.996887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcfa75-8d83-42ae-915f-5bc56e31a1fd admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:18.003770 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-59dcfa75-8d83-42ae-915f-5bc56e31a1fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:18.043224 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcfa75-8d83-42ae-915f-5bc56e31a1fd admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:18.140348 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83db35-c960-45ca-932b-b862929049b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:18.141023 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83db35-c960-45ca-932b-b862929049b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:18.141349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83db35-c960-45ca-932b-b862929049b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:18.141616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83db35-c960-45ca-932b-b862929049b0 admin admin] PATH_INFO: `/v3/projects/8b25e8a3ea4f4399ab7e5baee4b11465/users/d11cf92a1a594e7084aa52d5d4e1c690/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:18.168177 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83db35-c960-45ca-932b-b862929049b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b25e8a3ea4f4399ab7e5baee4b11465, user_id=d11cf92a1a594e7084aa52d5d4e1c690, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:18.169266 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: WARNING py.warnings [None req-9a83db35-c960-45ca-932b-b862929049b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:21:18.169266 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: warnings.warn(msg) Sep 19 11:21:18.169266 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: Sep 19 11:21:18.171147 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83db35-c960-45ca-932b-b862929049b0 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:18.289304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 743/1501] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:21:17 2022] PUT /identity/v3/projects/8b25e8a3ea4f4399ab7e5baee4b11465/users/d11cf92a1a594e7084aa52d5d4e1c690/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:21:18.301656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a145f-5877-4b7c-99cf-95cab10197ed None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:18.301982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a145f-5877-4b7c-99cf-95cab10197ed None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:18.302362 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a145f-5877-4b7c-99cf-95cab10197ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:18.364273 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-415a145f-5877-4b7c-99cf-95cab10197ed None None] MFA Rules not processed for user `d11cf92a1a594e7084aa52d5d4e1c690`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:21:18.365488 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-415a145f-5877-4b7c-99cf-95cab10197ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:18.459236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 759/1502] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:21:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:21:18.476779 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9604d-737a-4f21-bb05-2b1f5b6b03cd None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:18.477074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9604d-737a-4f21-bb05-2b1f5b6b03cd None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:18.477352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9604d-737a-4f21-bb05-2b1f5b6b03cd None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:18.478752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 744/1503] 10.222.0.78 () {54 vars in 967 bytes} [Mon Sep 19 11:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:21:18.483640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583191a-1544-4df8-bccc-7a4bb95bf06e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:18.483929 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583191a-1544-4df8-bccc-7a4bb95bf06e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:18.484206 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583191a-1544-4df8-bccc-7a4bb95bf06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:18.524348 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:21:18.574005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-2583191a-1544-4df8-bccc-7a4bb95bf06e None None] MFA Rules not processed for user `1ab94f6e08644fa7a76b9057e46d122d`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:21:18.575596 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2583191a-1544-4df8-bccc-7a4bb95bf06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:18.659904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 760/1504] 10.222.0.78 () {58 vars in 1064 bytes} [Mon Sep 19 11:21:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:21:18.668905 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9604d-737a-4f21-bb05-2b1f5b6b03cd None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:18.670530 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-87f9604d-737a-4f21-bb05-2b1f5b6b03cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:18.725178 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:21:18.787591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9604d-737a-4f21-bb05-2b1f5b6b03cd None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:18.873529 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f148b-733b-4985-bb47-fb6fd646954c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:18.874296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f148b-733b-4985-bb47-fb6fd646954c service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:18.874504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f148b-733b-4985-bb47-fb6fd646954c service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:18.874613 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f148b-733b-4985-bb47-fb6fd646954c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:18.876290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-786f148b-733b-4985-bb47-fb6fd646954c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:18.977656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f148b-733b-4985-bb47-fb6fd646954c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:18.979088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f148b-733b-4985-bb47-fb6fd646954c service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:18.982558 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:21:18.997829 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 745/1505] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:21:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:21:19.044187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bbcc13-048b-4634-9f1f-0b1358ae11af None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:19.044548 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bbcc13-048b-4634-9f1f-0b1358ae11af None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:19.044852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bbcc13-048b-4634-9f1f-0b1358ae11af None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:19.046110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 761/1506] 10.222.0.78 () {52 vars in 830 bytes} [Mon Sep 19 11:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:21:19.049736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f148b-733b-4985-bb47-fb6fd646954c service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:19.050973 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-786f148b-733b-4985-bb47-fb6fd646954c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:19.154314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f148b-733b-4985-bb47-fb6fd646954c service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:19.258425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8fb1c-f3dd-428b-8288-3edbba5ffa6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:19.259056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8fb1c-f3dd-428b-8288-3edbba5ffa6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:19.259369 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8fb1c-f3dd-428b-8288-3edbba5ffa6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:19.259642 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8fb1c-f3dd-428b-8288-3edbba5ffa6b admin admin] PATH_INFO: `/v3/users/d11cf92a1a594e7084aa52d5d4e1c690` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:19.272738 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8fb1c-f3dd-428b-8288-3edbba5ffa6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d11cf92a1a594e7084aa52d5d4e1c690)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:19.274322 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: WARNING py.warnings [None req-12c8fb1c-f3dd-428b-8288-3edbba5ffa6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:21:19.274322 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: warnings.warn(msg) Sep 19 11:21:19.274322 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: Sep 19 11:21:19.275599 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8fb1c-f3dd-428b-8288-3edbba5ffa6b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:19.287000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:21:19.502114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-12c8fb1c-f3dd-428b-8288-3edbba5ffa6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd11cf92a1a594e7084aa52d5d4e1c690'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:21:19.526401 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-12c8fb1c-f3dd-428b-8288-3edbba5ffa6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd11cf92a1a594e7084aa52d5d4e1c690'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:21:19.540402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-12c8fb1c-f3dd-428b-8288-3edbba5ffa6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd11cf92a1a594e7084aa52d5d4e1c690'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:21:19.612498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-12c8fb1c-f3dd-428b-8288-3edbba5ffa6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd11cf92a1a594e7084aa52d5d4e1c690'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:21:19.612937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-12c8fb1c-f3dd-428b-8288-3edbba5ffa6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd11cf92a1a594e7084aa52d5d4e1c690'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:21:19.619243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 746/1507] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:21:19 2022] DELETE /identity/v3/users/d11cf92a1a594e7084aa52d5d4e1c690 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:21:19.623661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ea465-9aa3-4483-8e40-5346990cd17d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:19.623947 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ea465-9aa3-4483-8e40-5346990cd17d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:19.624225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ea465-9aa3-4483-8e40-5346990cd17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:19.682234 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c81ea465-9aa3-4483-8e40-5346990cd17d None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:21:19.683490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c81ea465-9aa3-4483-8e40-5346990cd17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:19.760542 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:21:19.772626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 762/1508] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:21:19.782452 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8fb1c-f3dd-428b-8288-3edbba5ffa6b admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:19.783707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-12c8fb1c-f3dd-428b-8288-3edbba5ffa6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:19.884310 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8fb1c-f3dd-428b-8288-3edbba5ffa6b admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:19.939604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:21:19.960813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cdfb42-8faf-43de-9463-35b51b4a393a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:19.961437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cdfb42-8faf-43de-9463-35b51b4a393a service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:19.961716 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cdfb42-8faf-43de-9463-35b51b4a393a service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:19.961986 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cdfb42-8faf-43de-9463-35b51b4a393a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:19.963638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b2cdfb42-8faf-43de-9463-35b51b4a393a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:20.075659 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cdfb42-8faf-43de-9463-35b51b4a393a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:20.077000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cdfb42-8faf-43de-9463-35b51b4a393a service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:20.094611 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 747/1509] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:21:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:21:20.574718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ea465-9aa3-4483-8e40-5346990cd17d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:20.576138 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c81ea465-9aa3-4483-8e40-5346990cd17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:20.588146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ea465-9aa3-4483-8e40-5346990cd17d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:20.686220 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1d38a-0ac9-4c05-bd8b-a2c2e1094dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:20.686916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1d38a-0ac9-4c05-bd8b-a2c2e1094dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:20.687126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1d38a-0ac9-4c05-bd8b-a2c2e1094dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:20.687538 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1d38a-0ac9-4c05-bd8b-a2c2e1094dbe admin admin] PATH_INFO: `/v3/projects/8b25e8a3ea4f4399ab7e5baee4b11465` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:20.698813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1d38a-0ac9-4c05-bd8b-a2c2e1094dbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b25e8a3ea4f4399ab7e5baee4b11465)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:20.699862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: WARNING py.warnings [None req-73f1d38a-0ac9-4c05-bd8b-a2c2e1094dbe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:21:20.699862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: warnings.warn(msg) Sep 19 11:21:20.699862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: Sep 19 11:21:20.701066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1d38a-0ac9-4c05-bd8b-a2c2e1094dbe admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:20.816626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-73f1d38a-0ac9-4c05-bd8b-a2c2e1094dbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b25e8a3ea4f4399ab7e5baee4b11465', 'request_id': 'req-73f1d38a-0ac9-4c05-bd8b-a2c2e1094dbe'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:21:20.874290 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-73f1d38a-0ac9-4c05-bd8b-a2c2e1094dbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b25e8a3ea4f4399ab7e5baee4b11465', 'request_id': 'req-73f1d38a-0ac9-4c05-bd8b-a2c2e1094dbe'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:21:20.888686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-73f1d38a-0ac9-4c05-bd8b-a2c2e1094dbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b25e8a3ea4f4399ab7e5baee4b11465', 'request_id': 'req-73f1d38a-0ac9-4c05-bd8b-a2c2e1094dbe'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:21:20.911985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-73f1d38a-0ac9-4c05-bd8b-a2c2e1094dbe admin admin] The token cache is being invalidate because project 8b25e8a3ea4f4399ab7e5baee4b11465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:21:20.912279 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-73f1d38a-0ac9-4c05-bd8b-a2c2e1094dbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:21:20.912714 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-73f1d38a-0ac9-4c05-bd8b-a2c2e1094dbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:21:20.914636 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 763/1510] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:21:20 2022] DELETE /identity/v3/projects/8b25e8a3ea4f4399ab7e5baee4b11465 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:21:30.092464 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=62 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:21:30.912289 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:22:54.794293 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dbbec-a938-4caa-aae8-15776a0987ce None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:22:54.794906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dbbec-a938-4caa-aae8-15776a0987ce None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:22:54.794906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dbbec-a938-4caa-aae8-15776a0987ce None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:22:54.796209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:22:54.800995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811ecdc-284e-4b96-a0e9-d90bbcd852f0 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:22:54.801496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811ecdc-284e-4b96-a0e9-d90bbcd852f0 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:22:54.801600 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811ecdc-284e-4b96-a0e9-d90bbcd852f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:22:54.883251 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0811ecdc-284e-4b96-a0e9-d90bbcd852f0 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:22:54.884482 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0811ecdc-284e-4b96-a0e9-d90bbcd852f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:22:54.988467 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:22:55.141693 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dbbec-a938-4caa-aae8-15776a0987ce None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:22:55.143898 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2a2dbbec-a938-4caa-aae8-15776a0987ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:22:55.261531 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dbbec-a938-4caa-aae8-15776a0987ce None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:22:55.266933 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb32d51-6d38-478f-9d40-56f86cf11d2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:22:55.267570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb32d51-6d38-478f-9d40-56f86cf11d2c service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:22:55.267847 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb32d51-6d38-478f-9d40-56f86cf11d2c service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:22:55.268117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb32d51-6d38-478f-9d40-56f86cf11d2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:22:55.276483 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb32d51-6d38-478f-9d40-56f86cf11d2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:22:55.278058 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb32d51-6d38-478f-9d40-56f86cf11d2c service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:22:55.287795 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 749/1513] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:22:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:22:56.382432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d98a3-283e-412c-a276-8a1c7b00ce5e None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:22:56.382962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d98a3-283e-412c-a276-8a1c7b00ce5e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:22:56.382962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d98a3-283e-412c-a276-8a1c7b00ce5e None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:22:56.384283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:22:56.390736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c55b6-c591-4aaa-8e04-13dfbdd1ee6f None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:22:56.391188 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c55b6-c591-4aaa-8e04-13dfbdd1ee6f None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:22:56.391300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c55b6-c591-4aaa-8e04-13dfbdd1ee6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:22:56.412277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-fc2c55b6-c591-4aaa-8e04-13dfbdd1ee6f None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:22:56.413593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-fc2c55b6-c591-4aaa-8e04-13dfbdd1ee6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:22:56.421657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:22:56.578741 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d98a3-283e-412c-a276-8a1c7b00ce5e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:22:56.593769 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d98a3-283e-412c-a276-8a1c7b00ce5e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:22:56.599787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6be1aa-a960-4c11-963c-123de9cb97fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:22:56.600673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6be1aa-a960-4c11-963c-123de9cb97fa service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:22:56.600673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6be1aa-a960-4c11-963c-123de9cb97fa service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:22:56.600955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6be1aa-a960-4c11-963c-123de9cb97fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:22:56.609653 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6be1aa-a960-4c11-963c-123de9cb97fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:22:56.611608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6be1aa-a960-4c11-963c-123de9cb97fa service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:22:56.622284 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 766/1516] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:22:56.862461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea8164-0fed-4e18-9cbb-7606feecb75d None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:22:56.862745 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea8164-0fed-4e18-9cbb-7606feecb75d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:22:56.863045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea8164-0fed-4e18-9cbb-7606feecb75d None None] PATH_INFO: `/v3` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:22:56.864962 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 751/1517] 10.222.0.83 () {54 vars in 925 bytes} [Mon Sep 19 11:22:56 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:22:56.871160 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8d453-cb01-4fee-b071-3a8bbf6ba84d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:22:56.871489 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8d453-cb01-4fee-b071-3a8bbf6ba84d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:22:56.871765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8d453-cb01-4fee-b071-3a8bbf6ba84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:22:56.933670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f2e8d453-cb01-4fee-b071-3a8bbf6ba84d None None] MFA Rules not processed for user `0427cdd3905a422cab4fed0aac187ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:22:56.934845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f2e8d453-cb01-4fee-b071-3a8bbf6ba84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:22:56.951369 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:22:57.005415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 767/1518] 10.222.0.83 () {58 vars in 1015 bytes} [Mon Sep 19 11:22:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:00.859507 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5b950-d86c-4896-a035-62581b632798 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:00.859966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5b950-d86c-4896-a035-62581b632798 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:00.860080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5b950-d86c-4896-a035-62581b632798 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:00.861378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 752/1519] 10.222.0.78 () {54 vars in 961 bytes} [Mon Sep 19 11:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:23:00.866650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34174ee-4fda-4a6b-93f4-703b1066d423 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:00.867119 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34174ee-4fda-4a6b-93f4-703b1066d423 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:00.867239 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34174ee-4fda-4a6b-93f4-703b1066d423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:00.888206 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d34174ee-4fda-4a6b-93f4-703b1066d423 None None] MFA Rules not processed for user `0427cdd3905a422cab4fed0aac187ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:00.889456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d34174ee-4fda-4a6b-93f4-703b1066d423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:00.899289 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 768/1520] 10.222.0.78 () {58 vars in 1058 bytes} [Mon Sep 19 11:23:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:23:00.907079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5b950-d86c-4896-a035-62581b632798 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:00.920644 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5b950-d86c-4896-a035-62581b632798 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:00.925652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600b558-135e-460c-9c15-1a6274be2b1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:00.926316 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600b558-135e-460c-9c15-1a6274be2b1b service placement] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:00.926597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600b558-135e-460c-9c15-1a6274be2b1b service placement] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:00.926868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600b558-135e-460c-9c15-1a6274be2b1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:00.935241 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600b558-135e-460c-9c15-1a6274be2b1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:00.936815 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600b558-135e-460c-9c15-1a6274be2b1b service placement] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:00.946948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 753/1521] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:23:08.131868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28733d8-603f-4093-bbd4-7e90b4c13bfc None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:08.132341 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28733d8-603f-4093-bbd4-7e90b4c13bfc None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:08.132451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28733d8-603f-4093-bbd4-7e90b4c13bfc None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:08.133764 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:23:08.140042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30321dfe-5b3d-462e-9f89-992e187e3ca4 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:08.140488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30321dfe-5b3d-462e-9f89-992e187e3ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:08.140595 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30321dfe-5b3d-462e-9f89-992e187e3ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:08.159583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-30321dfe-5b3d-462e-9f89-992e187e3ca4 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:08.160793 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-30321dfe-5b3d-462e-9f89-992e187e3ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:08.169081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:08.323485 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28733d8-603f-4093-bbd4-7e90b4c13bfc None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:08.337686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28733d8-603f-4093-bbd4-7e90b4c13bfc None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:08.343077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3aab0-783d-4daa-9f29-f20332ffafb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:08.343697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3aab0-783d-4daa-9f29-f20332ffafb3 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:08.343973 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3aab0-783d-4daa-9f29-f20332ffafb3 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:08.344242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3aab0-783d-4daa-9f29-f20332ffafb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:08.352994 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3aab0-783d-4daa-9f29-f20332ffafb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:08.354582 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3aab0-783d-4daa-9f29-f20332ffafb3 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:08.364000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 770/1524] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:09.455143 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266aae5-5712-425e-8277-d8aeba010dc7 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:09.455729 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266aae5-5712-425e-8277-d8aeba010dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:09.455729 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266aae5-5712-425e-8277-d8aeba010dc7 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:09.457052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:23:09.464234 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d669b42-92f5-46b6-86db-989c535325d4 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:09.464792 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d669b42-92f5-46b6-86db-989c535325d4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:09.464792 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d669b42-92f5-46b6-86db-989c535325d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:09.485138 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-5d669b42-92f5-46b6-86db-989c535325d4 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:09.486386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5d669b42-92f5-46b6-86db-989c535325d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:09.495505 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:09.664835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266aae5-5712-425e-8277-d8aeba010dc7 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:09.679634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266aae5-5712-425e-8277-d8aeba010dc7 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:09.685059 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d6143-4366-43b4-9bc7-92463879b569 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:09.685659 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d6143-4366-43b4-9bc7-92463879b569 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:09.685942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d6143-4366-43b4-9bc7-92463879b569 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:09.686218 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d6143-4366-43b4-9bc7-92463879b569 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:09.694705 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d6143-4366-43b4-9bc7-92463879b569 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:09.696317 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d6143-4366-43b4-9bc7-92463879b569 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:09.705707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 756/1527] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:13.993738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d23dbb-68e8-431a-8c80-ba82766c6ae3 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:13.994389 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d23dbb-68e8-431a-8c80-ba82766c6ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:13.994389 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d23dbb-68e8-431a-8c80-ba82766c6ae3 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:13.995807 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:23:14.001030 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd28752-b637-4ce9-8306-0cbad94130ea None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:14.001489 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd28752-b637-4ce9-8306-0cbad94130ea None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:14.001596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd28752-b637-4ce9-8306-0cbad94130ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:14.021484 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-1cd28752-b637-4ce9-8306-0cbad94130ea None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:14.022691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1cd28752-b637-4ce9-8306-0cbad94130ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:14.030514 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:14.324153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d23dbb-68e8-431a-8c80-ba82766c6ae3 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:14.325967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-28d23dbb-68e8-431a-8c80-ba82766c6ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:14.411080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d23dbb-68e8-431a-8c80-ba82766c6ae3 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:14.415520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30477a1-b875-4105-83ca-204f5f5b90b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:14.416138 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30477a1-b875-4105-83ca-204f5f5b90b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:14.416424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30477a1-b875-4105-83ca-204f5f5b90b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:14.416705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30477a1-b875-4105-83ca-204f5f5b90b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:14.426788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30477a1-b875-4105-83ca-204f5f5b90b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:14.428697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30477a1-b875-4105-83ca-204f5f5b90b2 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:14.438664 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 773/1530] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:16.583976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e11e9-0336-4223-8cac-e45d7e73f071 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:16.585035 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e11e9-0336-4223-8cac-e45d7e73f071 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:16.585035 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e11e9-0336-4223-8cac-e45d7e73f071 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:16.586362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:23:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:23:16.591712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dbc0d-03ef-4631-82a0-ffba081a98da None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:16.592176 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dbc0d-03ef-4631-82a0-ffba081a98da None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:16.592277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dbc0d-03ef-4631-82a0-ffba081a98da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:16.615280 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-770dbc0d-03ef-4631-82a0-ffba081a98da None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:16.615280 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-770dbc0d-03ef-4631-82a0-ffba081a98da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:16.621961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:16.912655 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e11e9-0336-4223-8cac-e45d7e73f071 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:16.923153 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e11e9-0336-4223-8cac-e45d7e73f071 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:16.926628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d8b20-0899-46bf-b196-46e5b1b616c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:16.927249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d8b20-0899-46bf-b196-46e5b1b616c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:16.928130 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d8b20-0899-46bf-b196-46e5b1b616c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:16.928405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d8b20-0899-46bf-b196-46e5b1b616c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:16.937158 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d8b20-0899-46bf-b196-46e5b1b616c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:16.938559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d8b20-0899-46bf-b196-46e5b1b616c2 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:16.948621 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 759/1533] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:20.849861 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0e564-d1fb-421b-9bc7-0879c01c5a61 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.850365 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0e564-d1fb-421b-9bc7-0879c01c5a61 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.857201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0e564-d1fb-421b-9bc7-0879c01c5a61 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.857201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:23:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:23:20.857437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf35b9-202a-4a12-97f8-1318800545a0 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.857824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf35b9-202a-4a12-97f8-1318800545a0 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.857927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf35b9-202a-4a12-97f8-1318800545a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.866118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=81 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:20.878293 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-4dcf35b9-202a-4a12-97f8-1318800545a0 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:20.879660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4dcf35b9-202a-4a12-97f8-1318800545a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:20.887502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:21.191553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0e564-d1fb-421b-9bc7-0879c01c5a61 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:21.193240 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-19b0e564-d1fb-421b-9bc7-0879c01c5a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:21.258964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:21.272383 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0e564-d1fb-421b-9bc7-0879c01c5a61 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:21.284472 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4958f51-803d-496d-aa86-b3acc6a1bc97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:21.284472 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4958f51-803d-496d-aa86-b3acc6a1bc97 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.284472 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4958f51-803d-496d-aa86-b3acc6a1bc97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.284472 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4958f51-803d-496d-aa86-b3acc6a1bc97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.289999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4958f51-803d-496d-aa86-b3acc6a1bc97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:21.291484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4958f51-803d-496d-aa86-b3acc6a1bc97 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:21.301431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 776/1536] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:23.482964 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1a25f-ba3b-4e5b-968b-7bf252c5cb01 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:23.483507 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1a25f-ba3b-4e5b-968b-7bf252c5cb01 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:23.483507 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1a25f-ba3b-4e5b-968b-7bf252c5cb01 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:23.484794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:23:23.490502 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6697f52-7c41-4264-b727-96981fd00936 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:23.490889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6697f52-7c41-4264-b727-96981fd00936 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:23.491051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6697f52-7c41-4264-b727-96981fd00936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:23.511047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a6697f52-7c41-4264-b727-96981fd00936 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:23.512282 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a6697f52-7c41-4264-b727-96981fd00936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:23.520152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:23.823444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1a25f-ba3b-4e5b-968b-7bf252c5cb01 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:23.825158 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-6bc1a25f-ba3b-4e5b-968b-7bf252c5cb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:23.838030 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1a25f-ba3b-4e5b-968b-7bf252c5cb01 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:23.841626 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba6490-456b-4f14-830b-10be3302448f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:23.842227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba6490-456b-4f14-830b-10be3302448f service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:23.842507 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba6490-456b-4f14-830b-10be3302448f service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:23.842776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba6490-456b-4f14-830b-10be3302448f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:23.853187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba6490-456b-4f14-830b-10be3302448f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:23.854604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba6490-456b-4f14-830b-10be3302448f service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:23.863904 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 762/1539] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:33.505585 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=11 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:33.859756 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=8 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:49.600051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1077c-9af5-4f6e-aa13-7de83cacea3e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:49.608866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1077c-9af5-4f6e-aa13-7de83cacea3e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:49.608866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1077c-9af5-4f6e-aa13-7de83cacea3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:49.628805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-69d1077c-9af5-4f6e-aa13-7de83cacea3e None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:49.630935 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-69d1077c-9af5-4f6e-aa13-7de83cacea3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:49.644838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 778/1540] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:49.654727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6961d9-7782-4ee7-82e8-859cc74c15fb None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:49.655326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6961d9-7782-4ee7-82e8-859cc74c15fb None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:49.655326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6961d9-7782-4ee7-82e8-859cc74c15fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:49.684256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-df6961d9-7782-4ee7-82e8-859cc74c15fb None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:49.685597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-df6961d9-7782-4ee7-82e8-859cc74c15fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:49.696720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 763/1541] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:49.701440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1077c-9af5-4f6e-aa13-7de83cacea3e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:49.717330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1077c-9af5-4f6e-aa13-7de83cacea3e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:49.723121 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e50da1-26ce-43b2-803e-c092a0adb4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:49.724052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50da1-26ce-43b2-803e-c092a0adb4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:49.724561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50da1-26ce-43b2-803e-c092a0adb4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:49.725171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50da1-26ce-43b2-803e-c092a0adb4b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:49.726051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e50da1-26ce-43b2-803e-c092a0adb4b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:49.726592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e50da1-26ce-43b2-803e-c092a0adb4b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:49.728547 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e50da1-26ce-43b2-803e-c092a0adb4b5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:49.746872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 779/1542] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:49.755318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6961d9-7782-4ee7-82e8-859cc74c15fb None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:49.769847 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6961d9-7782-4ee7-82e8-859cc74c15fb None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:49.780307 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2feef-e759-459b-b210-49cd36581698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:49.780955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2feef-e759-459b-b210-49cd36581698 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:49.781235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2feef-e759-459b-b210-49cd36581698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:49.781510 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2feef-e759-459b-b210-49cd36581698 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:49.782202 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2feef-e759-459b-b210-49cd36581698 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:49.784180 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2feef-e759-459b-b210-49cd36581698 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:49.937809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 764/1543] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:49 2022] POST /identity/v3/projects => generated 391 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:49.942035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e50da1-26ce-43b2-803e-c092a0adb4b5 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:49.959995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e50da1-26ce-43b2-803e-c092a0adb4b5 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:50.026495 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cefdad-506d-4c92-ad71-0867d3917cfb None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.026765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cefdad-506d-4c92-ad71-0867d3917cfb None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.027044 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cefdad-506d-4c92-ad71-0867d3917cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.055392 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f440563-c37c-4f9d-aefe-843eea2ef386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:50.060269 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f440563-c37c-4f9d-aefe-843eea2ef386 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.060652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f440563-c37c-4f9d-aefe-843eea2ef386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.060843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f440563-c37c-4f9d-aefe-843eea2ef386 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.061580 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f440563-c37c-4f9d-aefe-843eea2ef386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:50.063703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f440563-c37c-4f9d-aefe-843eea2ef386 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:50.069234 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-b5cefdad-506d-4c92-ad71-0867d3917cfb None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:50.072152 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b5cefdad-506d-4c92-ad71-0867d3917cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.089567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 765/1544] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:50.092280 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4504cb-03b7-477b-8b46-c57bff998eba None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.093122 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4504cb-03b7-477b-8b46-c57bff998eba None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.093642 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4504cb-03b7-477b-8b46-c57bff998eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.139857 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-2c4504cb-03b7-477b-8b46-c57bff998eba None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:50.141334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2c4504cb-03b7-477b-8b46-c57bff998eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.156568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 766/1545] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:50.158399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844420a2-16e6-48ea-a25e-05ea9ea83472 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.158759 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844420a2-16e6-48ea-a25e-05ea9ea83472 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.159125 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844420a2-16e6-48ea-a25e-05ea9ea83472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.190885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-844420a2-16e6-48ea-a25e-05ea9ea83472 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:50.191590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 780/1546] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:49 2022] POST /identity/v3/users => generated 296 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:50.192887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcff937-00c4-40df-ac3b-ae07870bb284 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.193139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-844420a2-16e6-48ea-a25e-05ea9ea83472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.193263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcff937-00c4-40df-ac3b-ae07870bb284 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.193505 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcff937-00c4-40df-ac3b-ae07870bb284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.211183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 767/1547] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:50.212473 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844420a2-16e6-48ea-a25e-05ea9ea83472 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:50.232768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844420a2-16e6-48ea-a25e-05ea9ea83472 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:50.244101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915fc98-d79e-4f6d-8d6e-64d96f8a75a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:50.244743 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915fc98-d79e-4f6d-8d6e-64d96f8a75a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.244929 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-3dcff937-00c4-40df-ac3b-ae07870bb284 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:50.245050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915fc98-d79e-4f6d-8d6e-64d96f8a75a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.245288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915fc98-d79e-4f6d-8d6e-64d96f8a75a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.246646 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915fc98-d79e-4f6d-8d6e-64d96f8a75a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:50.246769 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3dcff937-00c4-40df-ac3b-ae07870bb284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.252937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915fc98-d79e-4f6d-8d6e-64d96f8a75a0 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:50.270552 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 768/1548] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:50.271895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5e506-4af4-4a61-9aee-ba5797666894 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.272183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5e506-4af4-4a61-9aee-ba5797666894 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.272462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5e506-4af4-4a61-9aee-ba5797666894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.281446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 781/1549] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:50.282142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcff937-00c4-40df-ac3b-ae07870bb284 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:50.301706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcff937-00c4-40df-ac3b-ae07870bb284 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:50.310750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-46a5e506-4af4-4a61-9aee-ba5797666894 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:50.312217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-46a5e506-4af4-4a61-9aee-ba5797666894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.314904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1540ab6-569d-445b-a11f-f885dc839170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:50.316021 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1540ab6-569d-445b-a11f-f885dc839170 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.322675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1540ab6-569d-445b-a11f-f885dc839170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.322675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1540ab6-569d-445b-a11f-f885dc839170 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.322675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1540ab6-569d-445b-a11f-f885dc839170 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:50.322675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1540ab6-569d-445b-a11f-f885dc839170 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:50.322675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1540ab6-569d-445b-a11f-f885dc839170 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:50.323090 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 769/1550] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:50.323209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76199762-2277-45ec-aff9-a3ad0de9ff88 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.323597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76199762-2277-45ec-aff9-a3ad0de9ff88 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.323881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76199762-2277-45ec-aff9-a3ad0de9ff88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.342447 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 782/1551] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:50.343962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1540ab6-569d-445b-a11f-f885dc839170 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:50.352385 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-76199762-2277-45ec-aff9-a3ad0de9ff88 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:50.353602 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-76199762-2277-45ec-aff9-a3ad0de9ff88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.354692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1540ab6-569d-445b-a11f-f885dc839170 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:50.359424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c793da-2f3d-41a4-8315-ba328809ac4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:50.360032 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c793da-2f3d-41a4-8315-ba328809ac4e admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.360324 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c793da-2f3d-41a4-8315-ba328809ac4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.360595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c793da-2f3d-41a4-8315-ba328809ac4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.361236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c793da-2f3d-41a4-8315-ba328809ac4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:50.361522 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 770/1552] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:50.362439 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76199762-2277-45ec-aff9-a3ad0de9ff88 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:50.362964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c793da-2f3d-41a4-8315-ba328809ac4e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:50.375262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 783/1553] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:50.375640 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76199762-2277-45ec-aff9-a3ad0de9ff88 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:50.376517 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda7bb3-00c5-427a-b5be-b693e01334a4 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.376804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda7bb3-00c5-427a-b5be-b693e01334a4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.377081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda7bb3-00c5-427a-b5be-b693e01334a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.383488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaf48f-e1c4-4993-95f7-aa61a8cefc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:50.384103 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaf48f-e1c4-4993-95f7-aa61a8cefc7b admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.384388 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaf48f-e1c4-4993-95f7-aa61a8cefc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.384660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaf48f-e1c4-4993-95f7-aa61a8cefc7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.385343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aaf48f-e1c4-4993-95f7-aa61a8cefc7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:50.385646 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aaf48f-e1c4-4993-95f7-aa61a8cefc7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:50.386663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: WARNING py.warnings [None req-66aaf48f-e1c4-4993-95f7-aa61a8cefc7b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:24:50.386663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: warnings.warn(msg) Sep 19 11:24:50.386663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: Sep 19 11:24:50.388194 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aaf48f-e1c4-4993-95f7-aa61a8cefc7b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:50.398135 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-9cda7bb3-00c5-427a-b5be-b693e01334a4 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:50.399430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9cda7bb3-00c5-427a-b5be-b693e01334a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.401275 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 771/1554] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:50.402546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050ff5f-d46e-4dc8-8a6c-444cfa224f01 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.402837 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050ff5f-d46e-4dc8-8a6c-444cfa224f01 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.403105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050ff5f-d46e-4dc8-8a6c-444cfa224f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.414073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 784/1555] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:50.415626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dda9e0-4f64-4af1-96d9-6a2f8a0b8f71 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.416016 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dda9e0-4f64-4af1-96d9-6a2f8a0b8f71 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.416396 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dda9e0-4f64-4af1-96d9-6a2f8a0b8f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.423341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-f050ff5f-d46e-4dc8-8a6c-444cfa224f01 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:50.424575 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f050ff5f-d46e-4dc8-8a6c-444cfa224f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.433175 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 772/1556] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:50.435088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805de36-6b67-4eae-8389-8d4d2bf4b9d5 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.435400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805de36-6b67-4eae-8389-8d4d2bf4b9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.435791 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805de36-6b67-4eae-8389-8d4d2bf4b9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.437699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c9dda9e0-4f64-4af1-96d9-6a2f8a0b8f71 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:50.438930 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c9dda9e0-4f64-4af1-96d9-6a2f8a0b8f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.448633 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 785/1557] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:50.449909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dda9e0-4f64-4af1-96d9-6a2f8a0b8f71 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:50.456103 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-4805de36-6b67-4eae-8389-8d4d2bf4b9d5 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:50.457301 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4805de36-6b67-4eae-8389-8d4d2bf4b9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.460173 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dda9e0-4f64-4af1-96d9-6a2f8a0b8f71 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:50.464928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf8d4f-2c6f-4b6a-8f1c-f3aec953f550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:50.474687 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf8d4f-2c6f-4b6a-8f1c-f3aec953f550 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.474877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 773/1558] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:50.474996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf8d4f-2c6f-4b6a-8f1c-f3aec953f550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.474996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf8d4f-2c6f-4b6a-8f1c-f3aec953f550 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.474996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf8d4f-2c6f-4b6a-8f1c-f3aec953f550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:50.475331 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b724df-b2b1-4bfb-89ab-3106efe813c9 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.475331 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b724df-b2b1-4bfb-89ab-3106efe813c9 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.475331 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b724df-b2b1-4bfb-89ab-3106efe813c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.475604 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: WARNING py.warnings [None req-38bf8d4f-2c6f-4b6a-8f1c-f3aec953f550 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:24:50.475604 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: warnings.warn(msg) Sep 19 11:24:50.475604 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: Sep 19 11:24:50.475604 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf8d4f-2c6f-4b6a-8f1c-f3aec953f550 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:50.492244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-b4b724df-b2b1-4bfb-89ab-3106efe813c9 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:50.493445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b4b724df-b2b1-4bfb-89ab-3106efe813c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.501725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 774/1559] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:50.502628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b724df-b2b1-4bfb-89ab-3106efe813c9 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:50.511840 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b724df-b2b1-4bfb-89ab-3106efe813c9 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:50.516883 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01160eb5-4db7-4495-808b-0fbfa1e5d009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:50.517503 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01160eb5-4db7-4495-808b-0fbfa1e5d009 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.517778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01160eb5-4db7-4495-808b-0fbfa1e5d009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.518048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01160eb5-4db7-4495-808b-0fbfa1e5d009 admin admin] PATH_INFO: `/v3/projects/7f4d504e02a645b6a5ebc78ed0bc5a41/users/d6fee905418f4d879da460b6758d4f23/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.532931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01160eb5-4db7-4495-808b-0fbfa1e5d009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f4d504e02a645b6a5ebc78ed0bc5a41, user_id=d6fee905418f4d879da460b6758d4f23, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:50.535604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01160eb5-4db7-4495-808b-0fbfa1e5d009 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:50.677698 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 786/1560] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/projects => generated 377 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:50.678622 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf8d4f-2c6f-4b6a-8f1c-f3aec953f550 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:50.689813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 775/1561] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:50 2022] PUT /identity/v3/projects/7f4d504e02a645b6a5ebc78ed0bc5a41/users/d6fee905418f4d879da460b6758d4f23/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:50.690988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67346044-8c64-458d-8399-4b8c32135fbb None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.691312 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67346044-8c64-458d-8399-4b8c32135fbb None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.691604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67346044-8c64-458d-8399-4b8c32135fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.712264 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-67346044-8c64-458d-8399-4b8c32135fbb None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:50.713585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-67346044-8c64-458d-8399-4b8c32135fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.744241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf8d4f-2c6f-4b6a-8f1c-f3aec953f550 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:50.753597 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:50.805661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700cc0c7-f071-4417-b5a4-68162f82ceb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:50.806306 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cc0c7-f071-4417-b5a4-68162f82ceb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.806589 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cc0c7-f071-4417-b5a4-68162f82ceb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.806864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cc0c7-f071-4417-b5a4-68162f82ceb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.807527 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700cc0c7-f071-4417-b5a4-68162f82ceb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:50.809552 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 776/1562] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:50.810421 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700cc0c7-f071-4417-b5a4-68162f82ceb7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:50.810808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e60ea3-431b-47a8-8bf2-4ac37e64f6bf None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.811087 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e60ea3-431b-47a8-8bf2-4ac37e64f6bf None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.811387 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e60ea3-431b-47a8-8bf2-4ac37e64f6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.831925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-d4e60ea3-431b-47a8-8bf2-4ac37e64f6bf None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:50.833126 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d4e60ea3-431b-47a8-8bf2-4ac37e64f6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.841252 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 777/1563] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:50.841881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e60ea3-431b-47a8-8bf2-4ac37e64f6bf None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:50.851411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e60ea3-431b-47a8-8bf2-4ac37e64f6bf None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:50.856427 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327a450-d330-4dfe-b6ca-b8e6d6d681cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:50.857011 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327a450-d330-4dfe-b6ca-b8e6d6d681cd admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.857278 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327a450-d330-4dfe-b6ca-b8e6d6d681cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.857543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327a450-d330-4dfe-b6ca-b8e6d6d681cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.858183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327a450-d330-4dfe-b6ca-b8e6d6d681cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:50.858488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327a450-d330-4dfe-b6ca-b8e6d6d681cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:50.860088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327a450-d330-4dfe-b6ca-b8e6d6d681cd admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:50.873129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 778/1564] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:50.874521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b1c5e-f22a-4013-aefb-f56100d47801 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.874811 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b1c5e-f22a-4013-aefb-f56100d47801 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.875089 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b1c5e-f22a-4013-aefb-f56100d47801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.882875 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:50.895577 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-f21b1c5e-f22a-4013-aefb-f56100d47801 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:50.899669 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 787/1565] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/projects => generated 373 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:50.899802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f21b1c5e-f22a-4013-aefb-f56100d47801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.899922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700cc0c7-f071-4417-b5a4-68162f82ceb7 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:50.958474 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 779/1566] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:50.959342 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b1c5e-f22a-4013-aefb-f56100d47801 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:50.964796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700cc0c7-f071-4417-b5a4-68162f82ceb7 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:50.969334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b1c5e-f22a-4013-aefb-f56100d47801 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:50.970400 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac18ec-fc59-4818-8edd-6765f5efe18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:50.971058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac18ec-fc59-4818-8edd-6765f5efe18e admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.971330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac18ec-fc59-4818-8edd-6765f5efe18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.971613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac18ec-fc59-4818-8edd-6765f5efe18e admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.972306 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac18ec-fc59-4818-8edd-6765f5efe18e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:50.972611 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac18ec-fc59-4818-8edd-6765f5efe18e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:50.974064 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5449236-1641-4070-8b50-3861cc51f9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:50.974206 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac18ec-fc59-4818-8edd-6765f5efe18e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:50.974668 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5449236-1641-4070-8b50-3861cc51f9cf admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.974949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5449236-1641-4070-8b50-3861cc51f9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.975247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5449236-1641-4070-8b50-3861cc51f9cf admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.976041 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5449236-1641-4070-8b50-3861cc51f9cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:50.977063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: WARNING py.warnings [None req-e5449236-1641-4070-8b50-3861cc51f9cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:24:50.977063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: warnings.warn(msg) Sep 19 11:24:50.977063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: Sep 19 11:24:50.978387 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5449236-1641-4070-8b50-3861cc51f9cf admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:50.994344 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 788/1567] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:50.994344 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cccea6-d15b-4feb-8840-909ba91ee348 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.994344 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cccea6-d15b-4feb-8840-909ba91ee348 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.994344 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cccea6-d15b-4feb-8840-909ba91ee348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.008252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-24cccea6-d15b-4feb-8840-909ba91ee348 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:51.009451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-24cccea6-d15b-4feb-8840-909ba91ee348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:51.016952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 789/1568] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:51.017776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cccea6-d15b-4feb-8840-909ba91ee348 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.028319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cccea6-d15b-4feb-8840-909ba91ee348 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.032979 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde42175-3825-432e-80d5-974e32ba0c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.033581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde42175-3825-432e-80d5-974e32ba0c43 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.033866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde42175-3825-432e-80d5-974e32ba0c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.034265 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde42175-3825-432e-80d5-974e32ba0c43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.034916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde42175-3825-432e-80d5-974e32ba0c43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:51.035248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde42175-3825-432e-80d5-974e32ba0c43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.036841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde42175-3825-432e-80d5-974e32ba0c43 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.049218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 790/1569] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:51.050026 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde42175-3825-432e-80d5-974e32ba0c43 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.059565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde42175-3825-432e-80d5-974e32ba0c43 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.064295 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a600e59-8026-4cd8-95f6-df33417c8f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.064869 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a600e59-8026-4cd8-95f6-df33417c8f7f admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.065151 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a600e59-8026-4cd8-95f6-df33417c8f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.065432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a600e59-8026-4cd8-95f6-df33417c8f7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.066063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a600e59-8026-4cd8-95f6-df33417c8f7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:51.066367 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a600e59-8026-4cd8-95f6-df33417c8f7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.067988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a600e59-8026-4cd8-95f6-df33417c8f7f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.080322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 791/1570] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:51.081206 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a600e59-8026-4cd8-95f6-df33417c8f7f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.091885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a600e59-8026-4cd8-95f6-df33417c8f7f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.098162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee141045-f047-44f8-92d8-6cafe32eceb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.098162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee141045-f047-44f8-92d8-6cafe32eceb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.098162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee141045-f047-44f8-92d8-6cafe32eceb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.098162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee141045-f047-44f8-92d8-6cafe32eceb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.098786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee141045-f047-44f8-92d8-6cafe32eceb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.100734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee141045-f047-44f8-92d8-6cafe32eceb8 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.140794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 780/1571] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:51.141792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5449236-1641-4070-8b50-3861cc51f9cf admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.152218 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5449236-1641-4070-8b50-3861cc51f9cf admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.157509 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f4a5f-4d74-485f-8b30-dbc7826dc8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.158112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f4a5f-4d74-485f-8b30-dbc7826dc8fb admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.158387 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f4a5f-4d74-485f-8b30-dbc7826dc8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.158658 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f4a5f-4d74-485f-8b30-dbc7826dc8fb admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.159444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f4a5f-4d74-485f-8b30-dbc7826dc8fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.161514 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f4a5f-4d74-485f-8b30-dbc7826dc8fb admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.239646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 792/1572] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:51 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:51.240386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee141045-f047-44f8-92d8-6cafe32eceb8 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.278419 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 781/1573] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:51 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:51.281231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f4a5f-4d74-485f-8b30-dbc7826dc8fb admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.282858 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:51.289941 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f4a5f-4d74-485f-8b30-dbc7826dc8fb admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.307994 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee141045-f047-44f8-92d8-6cafe32eceb8 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.312866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc20764-d050-498f-90e5-22399ddf1559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.313490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc20764-d050-498f-90e5-22399ddf1559 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.313777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc20764-d050-498f-90e5-22399ddf1559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.314047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc20764-d050-498f-90e5-22399ddf1559 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.314731 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc20764-d050-498f-90e5-22399ddf1559 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:51.315034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc20764-d050-498f-90e5-22399ddf1559 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.316691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc20764-d050-498f-90e5-22399ddf1559 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.330184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 793/1574] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:51.331567 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc20764-d050-498f-90e5-22399ddf1559 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.355237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4698d-9403-4107-8aa4-c3e8733a9270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.359405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4698d-9403-4107-8aa4-c3e8733a9270 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.359405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4698d-9403-4107-8aa4-c3e8733a9270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.359405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4698d-9403-4107-8aa4-c3e8733a9270 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.359405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b4698d-9403-4107-8aa4-c3e8733a9270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.359405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b4698d-9403-4107-8aa4-c3e8733a9270 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.390234 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc20764-d050-498f-90e5-22399ddf1559 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.393964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf038ab3-2f94-406e-9428-89dcc6ad8d85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.394572 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf038ab3-2f94-406e-9428-89dcc6ad8d85 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.394860 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf038ab3-2f94-406e-9428-89dcc6ad8d85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.395136 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf038ab3-2f94-406e-9428-89dcc6ad8d85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.405675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf038ab3-2f94-406e-9428-89dcc6ad8d85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.407125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf038ab3-2f94-406e-9428-89dcc6ad8d85 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.416827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 794/1575] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:51.417723 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf038ab3-2f94-406e-9428-89dcc6ad8d85 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.427635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf038ab3-2f94-406e-9428-89dcc6ad8d85 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.432381 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4465c3-9c9e-4a9b-a00a-4d2bafbdb21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.433000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4465c3-9c9e-4a9b-a00a-4d2bafbdb21c admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.433282 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4465c3-9c9e-4a9b-a00a-4d2bafbdb21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.433598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4465c3-9c9e-4a9b-a00a-4d2bafbdb21c admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.434247 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4465c3-9c9e-4a9b-a00a-4d2bafbdb21c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.436491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4465c3-9c9e-4a9b-a00a-4d2bafbdb21c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.486600 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 782/1576] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:51 2022] POST /identity/v3/projects => generated 365 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:51.486600 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4698d-9403-4107-8aa4-c3e8733a9270 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.537731 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:51.548029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4698d-9403-4107-8aa4-c3e8733a9270 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.553020 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd480f3-4e0a-4bcf-a9ec-d65a612d5149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.553644 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd480f3-4e0a-4bcf-a9ec-d65a612d5149 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.553923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd480f3-4e0a-4bcf-a9ec-d65a612d5149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.554192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd480f3-4e0a-4bcf-a9ec-d65a612d5149 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.554824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd480f3-4e0a-4bcf-a9ec-d65a612d5149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.556784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd480f3-4e0a-4bcf-a9ec-d65a612d5149 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.559010 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 795/1577] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:51 2022] POST /identity/v3/projects => generated 377 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:51.560065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4465c3-9c9e-4a9b-a00a-4d2bafbdb21c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.622932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4465c3-9c9e-4a9b-a00a-4d2bafbdb21c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.627796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e890ace-779e-4f68-89f5-8c01b7dc74bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.628409 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e890ace-779e-4f68-89f5-8c01b7dc74bd admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.628691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e890ace-779e-4f68-89f5-8c01b7dc74bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.628962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e890ace-779e-4f68-89f5-8c01b7dc74bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.629599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e890ace-779e-4f68-89f5-8c01b7dc74bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.631208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e890ace-779e-4f68-89f5-8c01b7dc74bd admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.642032 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 796/1578] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:51.645954 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e890ace-779e-4f68-89f5-8c01b7dc74bd admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.653144 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e890ace-779e-4f68-89f5-8c01b7dc74bd admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.657851 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989791c-9d42-46af-8373-e92a4fc270d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.658451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989791c-9d42-46af-8373-e92a4fc270d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.658734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989791c-9d42-46af-8373-e92a4fc270d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.659004 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989791c-9d42-46af-8373-e92a4fc270d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.659655 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989791c-9d42-46af-8373-e92a4fc270d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.661607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989791c-9d42-46af-8373-e92a4fc270d6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.749627 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 783/1579] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:51 2022] POST /identity/v3/projects => generated 385 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:51.750521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd480f3-4e0a-4bcf-a9ec-d65a612d5149 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.814706 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd480f3-4e0a-4bcf-a9ec-d65a612d5149 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.819372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7629018-9e32-475e-bcbd-9d4bf222107d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.819998 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7629018-9e32-475e-bcbd-9d4bf222107d admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.820269 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7629018-9e32-475e-bcbd-9d4bf222107d admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.820542 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7629018-9e32-475e-bcbd-9d4bf222107d admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.821185 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7629018-9e32-475e-bcbd-9d4bf222107d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.822784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7629018-9e32-475e-bcbd-9d4bf222107d admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.835899 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 784/1580] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:51.838710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7629018-9e32-475e-bcbd-9d4bf222107d admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.838850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 797/1581] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:51 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:51.838850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989791c-9d42-46af-8373-e92a4fc270d6 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.850567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7629018-9e32-475e-bcbd-9d4bf222107d admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.850942 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989791c-9d42-46af-8373-e92a4fc270d6 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.855472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b99f91-9104-4471-b531-88942c6e3ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.856623 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b99f91-9104-4471-b531-88942c6e3ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.856623 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b99f91-9104-4471-b531-88942c6e3ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.856812 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f44efa-bf19-4e94-a596-00ca71aa04c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.856917 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b99f91-9104-4471-b531-88942c6e3ae1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.857013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f44efa-bf19-4e94-a596-00ca71aa04c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.857275 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b99f91-9104-4471-b531-88942c6e3ae1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.857419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f44efa-bf19-4e94-a596-00ca71aa04c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.857594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f44efa-bf19-4e94-a596-00ca71aa04c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.858207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f44efa-bf19-4e94-a596-00ca71aa04c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.859192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b99f91-9104-4471-b531-88942c6e3ae1 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.860189 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f44efa-bf19-4e94-a596-00ca71aa04c5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.975020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 798/1582] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:51 2022] POST /identity/v3/users => generated 283 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:51.976269 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f44efa-bf19-4e94-a596-00ca71aa04c5 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.983310 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 785/1583] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:51 2022] POST /identity/v3/projects => generated 361 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:51.983310 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b99f91-9104-4471-b531-88942c6e3ae1 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.987225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f44efa-bf19-4e94-a596-00ca71aa04c5 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.999123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:52.050843 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b99f91-9104-4471-b531-88942c6e3ae1 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.052547 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44046168-4e8e-4c59-b923-c8484360a2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.053165 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44046168-4e8e-4c59-b923-c8484360a2de admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.053442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44046168-4e8e-4c59-b923-c8484360a2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.053713 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44046168-4e8e-4c59-b923-c8484360a2de admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.054347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44046168-4e8e-4c59-b923-c8484360a2de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.056267 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e72c7d-3b4b-400f-8f91-52342762e5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.056429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44046168-4e8e-4c59-b923-c8484360a2de admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.056888 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e72c7d-3b4b-400f-8f91-52342762e5c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.057167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e72c7d-3b4b-400f-8f91-52342762e5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.057441 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e72c7d-3b4b-400f-8f91-52342762e5c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.058086 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e72c7d-3b4b-400f-8f91-52342762e5c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.060365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e72c7d-3b4b-400f-8f91-52342762e5c2 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.071709 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 786/1584] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:52.072615 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e72c7d-3b4b-400f-8f91-52342762e5c2 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.082671 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e72c7d-3b4b-400f-8f91-52342762e5c2 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.098658 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab9957-1e72-4b54-b54a-25af3340f7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.098658 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab9957-1e72-4b54-b54a-25af3340f7e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.098658 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab9957-1e72-4b54-b54a-25af3340f7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.098658 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab9957-1e72-4b54-b54a-25af3340f7e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.099019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab9957-1e72-4b54-b54a-25af3340f7e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.099019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab9957-1e72-4b54-b54a-25af3340f7e0 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.195254 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 799/1585] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:51 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:52.195254 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44046168-4e8e-4c59-b923-c8484360a2de admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.203783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44046168-4e8e-4c59-b923-c8484360a2de admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.208715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e76e9-3e0c-441d-bb97-3bf23ce50eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.209339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e76e9-3e0c-441d-bb97-3bf23ce50eec admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.209623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e76e9-3e0c-441d-bb97-3bf23ce50eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.209899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e76e9-3e0c-441d-bb97-3bf23ce50eec admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.210555 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e76e9-3e0c-441d-bb97-3bf23ce50eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.220650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e76e9-3e0c-441d-bb97-3bf23ce50eec admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.223655 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 800/1586] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:52.224562 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e76e9-3e0c-441d-bb97-3bf23ce50eec admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.235634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 787/1587] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:52 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:52.235634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab9957-1e72-4b54-b54a-25af3340f7e0 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.241538 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e76e9-3e0c-441d-bb97-3bf23ce50eec admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.245268 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab9957-1e72-4b54-b54a-25af3340f7e0 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.249054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e337bb-66c6-4987-b9b4-38a8921d2aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.250372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e337bb-66c6-4987-b9b4-38a8921d2aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.250976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e337bb-66c6-4987-b9b4-38a8921d2aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.251408 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf974cee-580b-4112-a2f1-b2a99c86051b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.251716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e337bb-66c6-4987-b9b4-38a8921d2aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.252304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf974cee-580b-4112-a2f1-b2a99c86051b admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.252657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf974cee-580b-4112-a2f1-b2a99c86051b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.253014 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e337bb-66c6-4987-b9b4-38a8921d2aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.253169 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf974cee-580b-4112-a2f1-b2a99c86051b admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.253937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf974cee-580b-4112-a2f1-b2a99c86051b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.255287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e337bb-66c6-4987-b9b4-38a8921d2aa7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.256074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf974cee-580b-4112-a2f1-b2a99c86051b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.271943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 788/1588] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:52.272169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 801/1589] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:52.272272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf974cee-580b-4112-a2f1-b2a99c86051b admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.272372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e337bb-66c6-4987-b9b4-38a8921d2aa7 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.284157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf974cee-580b-4112-a2f1-b2a99c86051b admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.284409 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e337bb-66c6-4987-b9b4-38a8921d2aa7 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.289429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67f987-7921-4030-9c28-774b391a0bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.289943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8534e4-8cde-4a53-b83f-0542c2d909b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.290069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67f987-7921-4030-9c28-774b391a0bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.290302 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67f987-7921-4030-9c28-774b391a0bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.290574 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67f987-7921-4030-9c28-774b391a0bf1 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.290900 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8534e4-8cde-4a53-b83f-0542c2d909b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.291232 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67f987-7921-4030-9c28-774b391a0bf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.291336 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8534e4-8cde-4a53-b83f-0542c2d909b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.291665 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8534e4-8cde-4a53-b83f-0542c2d909b6 admin admin] PATH_INFO: `/v3/projects/089d6ade5a3d4e48868bc6cc190c6251/users/eec3ef0842b948deb7ecc46f7c56dd04/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.293711 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67f987-7921-4030-9c28-774b391a0bf1 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.308043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8534e4-8cde-4a53-b83f-0542c2d909b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=089d6ade5a3d4e48868bc6cc190c6251, user_id=eec3ef0842b948deb7ecc46f7c56dd04, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.309925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: WARNING py.warnings [None req-8c8534e4-8cde-4a53-b83f-0542c2d909b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:24:52.309925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: warnings.warn(msg) Sep 19 11:24:52.309925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: Sep 19 11:24:52.311957 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8534e4-8cde-4a53-b83f-0542c2d909b6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.381570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 789/1590] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:52 2022] POST /identity/v3/users => generated 281 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:52.382617 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67f987-7921-4030-9c28-774b391a0bf1 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.388382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:52.394189 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67f987-7921-4030-9c28-774b391a0bf1 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.399682 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d6560-f74a-4739-855c-448e5d7bbf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.400317 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d6560-f74a-4739-855c-448e5d7bbf90 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.400597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d6560-f74a-4739-855c-448e5d7bbf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.400873 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d6560-f74a-4739-855c-448e5d7bbf90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.401674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d6560-f74a-4739-855c-448e5d7bbf90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.403849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d6560-f74a-4739-855c-448e5d7bbf90 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.414729 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 790/1591] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:52.415615 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d6560-f74a-4739-855c-448e5d7bbf90 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.416988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 802/1592] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:52 2022] PUT /identity/v3/projects/089d6ade5a3d4e48868bc6cc190c6251/users/eec3ef0842b948deb7ecc46f7c56dd04/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:52.418799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8534e4-8cde-4a53-b83f-0542c2d909b6 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.427792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d6560-f74a-4739-855c-448e5d7bbf90 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.490534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528ba64-69a7-4b7c-8d30-3d5985b5c1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.491382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528ba64-69a7-4b7c-8d30-3d5985b5c1e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.491685 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528ba64-69a7-4b7c-8d30-3d5985b5c1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.491962 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528ba64-69a7-4b7c-8d30-3d5985b5c1e6 admin admin] PATH_INFO: `/v3/projects/68f214f15a7b46f2a05c1d8ff0ea6692/users/c36f918242244a37ab06617ca0ec2eab/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.492613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8534e4-8cde-4a53-b83f-0542c2d909b6 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.498772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac8df6-b44d-4774-a814-8a558b59fb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.499506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac8df6-b44d-4774-a814-8a558b59fb5c admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.499861 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac8df6-b44d-4774-a814-8a558b59fb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.500591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac8df6-b44d-4774-a814-8a558b59fb5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.501325 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac8df6-b44d-4774-a814-8a558b59fb5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.503343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac8df6-b44d-4774-a814-8a558b59fb5c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.508033 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528ba64-69a7-4b7c-8d30-3d5985b5c1e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f214f15a7b46f2a05c1d8ff0ea6692, user_id=c36f918242244a37ab06617ca0ec2eab, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.510939 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528ba64-69a7-4b7c-8d30-3d5985b5c1e6 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.515226 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 803/1593] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:52.516150 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac8df6-b44d-4774-a814-8a558b59fb5c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.526158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac8df6-b44d-4774-a814-8a558b59fb5c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.531701 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50042ecf-06e3-47e4-8273-b88b562fab9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.532334 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50042ecf-06e3-47e4-8273-b88b562fab9f admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.532615 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50042ecf-06e3-47e4-8273-b88b562fab9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.532887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50042ecf-06e3-47e4-8273-b88b562fab9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.533528 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50042ecf-06e3-47e4-8273-b88b562fab9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.535244 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50042ecf-06e3-47e4-8273-b88b562fab9f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.546794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 804/1594] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:52.547726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50042ecf-06e3-47e4-8273-b88b562fab9f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.557394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50042ecf-06e3-47e4-8273-b88b562fab9f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.562018 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b8a5f-2d08-4b98-bf25-af3ec6967d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.562603 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b8a5f-2d08-4b98-bf25-af3ec6967d16 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.562934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b8a5f-2d08-4b98-bf25-af3ec6967d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.563179 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b8a5f-2d08-4b98-bf25-af3ec6967d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.563816 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b8a5f-2d08-4b98-bf25-af3ec6967d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.565516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b8a5f-2d08-4b98-bf25-af3ec6967d16 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.575953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 805/1595] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:52.576753 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b8a5f-2d08-4b98-bf25-af3ec6967d16 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.588132 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b8a5f-2d08-4b98-bf25-af3ec6967d16 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.593966 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ade77-2d56-44c4-9773-7a5f42737a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.594563 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ade77-2d56-44c4-9773-7a5f42737a08 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.594841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ade77-2d56-44c4-9773-7a5f42737a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.595108 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ade77-2d56-44c4-9773-7a5f42737a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.595778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ade77-2d56-44c4-9773-7a5f42737a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.597365 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ade77-2d56-44c4-9773-7a5f42737a08 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.610693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 806/1596] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:52.620409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 791/1597] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:52 2022] PUT /identity/v3/projects/68f214f15a7b46f2a05c1d8ff0ea6692/users/c36f918242244a37ab06617ca0ec2eab/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:52.620537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ade77-2d56-44c4-9773-7a5f42737a08 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.620635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ef2c3-580c-4952-a921-57c40a3f9b80 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.620635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ef2c3-580c-4952-a921-57c40a3f9b80 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.620635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ef2c3-580c-4952-a921-57c40a3f9b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.630923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ade77-2d56-44c4-9773-7a5f42737a08 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.654512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-a46ef2c3-580c-4952-a921-57c40a3f9b80 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:52.656053 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a46ef2c3-580c-4952-a921-57c40a3f9b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:52.656303 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:52.721533 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a04a4-a9d9-4db0-81ac-ea81807d50c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.722165 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a04a4-a9d9-4db0-81ac-ea81807d50c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.722450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a04a4-a9d9-4db0-81ac-ea81807d50c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.722729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a04a4-a9d9-4db0-81ac-ea81807d50c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.723410 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a04a4-a9d9-4db0-81ac-ea81807d50c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.727681 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a04a4-a9d9-4db0-81ac-ea81807d50c1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.727836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 792/1598] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:52.729598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ef2c3-580c-4952-a921-57c40a3f9b80 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.741492 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 807/1599] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:52.741492 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a04a4-a9d9-4db0-81ac-ea81807d50c1 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.741763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ef2c3-580c-4952-a921-57c40a3f9b80 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.744987 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59baaaf-0140-4d3f-8c6f-5309cd4bef46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.745601 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59baaaf-0140-4d3f-8c6f-5309cd4bef46 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.745881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59baaaf-0140-4d3f-8c6f-5309cd4bef46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.746197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59baaaf-0140-4d3f-8c6f-5309cd4bef46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.746858 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59baaaf-0140-4d3f-8c6f-5309cd4bef46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.748490 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59baaaf-0140-4d3f-8c6f-5309cd4bef46 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.749442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a04a4-a9d9-4db0-81ac-ea81807d50c1 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.755349 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f91a4-78b0-4e0c-a5a0-a5175f32ce9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.756092 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f91a4-78b0-4e0c-a5a0-a5175f32ce9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.767273 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f91a4-78b0-4e0c-a5a0-a5175f32ce9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.767273 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f91a4-78b0-4e0c-a5a0-a5175f32ce9e admin admin] PATH_INFO: `/v3/projects/e3a8ceb9271e4036a1bba2e45e1de8ac/users/42eb827ed7f440ecb60581821f686d39/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.767502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 793/1600] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:52.767502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59baaaf-0140-4d3f-8c6f-5309cd4bef46 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.770366 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59baaaf-0140-4d3f-8c6f-5309cd4bef46 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.773456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f91a4-78b0-4e0c-a5a0-a5175f32ce9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a8ceb9271e4036a1bba2e45e1de8ac, user_id=42eb827ed7f440ecb60581821f686d39, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.775124 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb17338-dc14-4dba-9e90-50a9726bc485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.775817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb17338-dc14-4dba-9e90-50a9726bc485 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.776100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb17338-dc14-4dba-9e90-50a9726bc485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.776221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f91a4-78b0-4e0c-a5a0-a5175f32ce9e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.776379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb17338-dc14-4dba-9e90-50a9726bc485 admin admin] PATH_INFO: `/v3/projects/7baafab3509c49fa96dcf385ff195aba/users/07c3aeab01d2454eb2b8c781b93f645d/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.794476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb17338-dc14-4dba-9e90-50a9726bc485 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7baafab3509c49fa96dcf385ff195aba, user_id=07c3aeab01d2454eb2b8c781b93f645d, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.799664 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb17338-dc14-4dba-9e90-50a9726bc485 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.849145 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 808/1601] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:52 2022] PUT /identity/v3/projects/e3a8ceb9271e4036a1bba2e45e1de8ac/users/42eb827ed7f440ecb60581821f686d39/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:52.849915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f91a4-78b0-4e0c-a5a0-a5175f32ce9e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.912440 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 794/1602] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:52 2022] PUT /identity/v3/projects/7baafab3509c49fa96dcf385ff195aba/users/07c3aeab01d2454eb2b8c781b93f645d/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:52.913726 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca4c3f-4a13-4772-a29d-b0041a06e9fd None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.914014 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca4c3f-4a13-4772-a29d-b0041a06e9fd None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.914326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca4c3f-4a13-4772-a29d-b0041a06e9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.921893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f91a4-78b0-4e0c-a5a0-a5175f32ce9e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.962325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-50ca4c3f-4a13-4772-a29d-b0041a06e9fd None None] MFA Rules not processed for user `c36f918242244a37ab06617ca0ec2eab`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:52.963677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-50ca4c3f-4a13-4772-a29d-b0041a06e9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:52.993748 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d23c9a-26c4-453a-a404-da9c8a29320f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.994564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d23c9a-26c4-453a-a404-da9c8a29320f admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.995398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d23c9a-26c4-453a-a404-da9c8a29320f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.995398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d23c9a-26c4-453a-a404-da9c8a29320f admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.996235 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d23c9a-26c4-453a-a404-da9c8a29320f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.996729 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:52.998024 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d23c9a-26c4-453a-a404-da9c8a29320f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.009195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 809/1603] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:53.010778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d23c9a-26c4-453a-a404-da9c8a29320f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.023420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e6d23c9a-26c4-453a-a404-da9c8a29320f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:53.051454 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 795/1604] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:53.054715 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca4c3f-4a13-4772-a29d-b0041a06e9fd None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.069561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:53.073863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca4c3f-4a13-4772-a29d-b0041a06e9fd None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.080435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c685901-f217-459b-9a06-419b7e78b7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.081121 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c685901-f217-459b-9a06-419b7e78b7b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.081399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c685901-f217-459b-9a06-419b7e78b7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.081724 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c685901-f217-459b-9a06-419b7e78b7b3 admin admin] PATH_INFO: `/v3/projects/4767407e0fb04faaaa3a520128aa88df/users/405b3189506a44bc93555c40917b4cbe/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.099868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c685901-f217-459b-9a06-419b7e78b7b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4767407e0fb04faaaa3a520128aa88df, user_id=405b3189506a44bc93555c40917b4cbe, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.102493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c685901-f217-459b-9a06-419b7e78b7b3 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.102730 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d23c9a-26c4-453a-a404-da9c8a29320f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.108472 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f019e1-2558-4625-ba2d-a2935a78a111 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.110589 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f019e1-2558-4625-ba2d-a2935a78a111 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.110877 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f019e1-2558-4625-ba2d-a2935a78a111 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.111147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f019e1-2558-4625-ba2d-a2935a78a111 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.121119 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f019e1-2558-4625-ba2d-a2935a78a111 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.122603 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f019e1-2558-4625-ba2d-a2935a78a111 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.132278 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 810/1605] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:53.133053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f019e1-2558-4625-ba2d-a2935a78a111 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.147380 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f019e1-2558-4625-ba2d-a2935a78a111 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.153018 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a7647-26dc-4a98-897c-80111c7137f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.153772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a7647-26dc-4a98-897c-80111c7137f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.154249 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a7647-26dc-4a98-897c-80111c7137f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.154827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a7647-26dc-4a98-897c-80111c7137f7 admin admin] PATH_INFO: `/v3/projects/0f6f09e176314205bbad48d0d0a382b3/users/4a4d4f493712488190cba8e9edcabf5b/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.170780 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a7647-26dc-4a98-897c-80111c7137f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f6f09e176314205bbad48d0d0a382b3, user_id=4a4d4f493712488190cba8e9edcabf5b, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.173187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a7647-26dc-4a98-897c-80111c7137f7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.242570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 796/1606] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:53 2022] PUT /identity/v3/projects/4767407e0fb04faaaa3a520128aa88df/users/405b3189506a44bc93555c40917b4cbe/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:53.244094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee8d56-31c0-4259-bfd7-ae45709ce06f None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.245385 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee8d56-31c0-4259-bfd7-ae45709ce06f None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.245385 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee8d56-31c0-4259-bfd7-ae45709ce06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.273488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-5bee8d56-31c0-4259-bfd7-ae45709ce06f None None] MFA Rules not processed for user `42eb827ed7f440ecb60581821f686d39`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:53.274744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5bee8d56-31c0-4259-bfd7-ae45709ce06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:53.314074 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 811/1607] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:53 2022] PUT /identity/v3/projects/0f6f09e176314205bbad48d0d0a382b3/users/4a4d4f493712488190cba8e9edcabf5b/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:53.319311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04f7e6-158d-4ab5-b4bc-c90bd85f3be9 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.319607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04f7e6-158d-4ab5-b4bc-c90bd85f3be9 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.319884 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04f7e6-158d-4ab5-b4bc-c90bd85f3be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.354799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-7e04f7e6-158d-4ab5-b4bc-c90bd85f3be9 None None] MFA Rules not processed for user `07c3aeab01d2454eb2b8c781b93f645d`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:53.356262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7e04f7e6-158d-4ab5-b4bc-c90bd85f3be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:53.431968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 812/1608] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:53.433157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04f7e6-158d-4ab5-b4bc-c90bd85f3be9 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.443651 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 797/1609] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:53.445100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c7845-8af6-494f-aa79-c50cdff3f023 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.445439 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c7845-8af6-494f-aa79-c50cdff3f023 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.445772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c7845-8af6-494f-aa79-c50cdff3f023 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.448142 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 798/1610] 10.222.0.78 () {54 vars in 965 bytes} [Mon Sep 19 11:24:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:24:53.450420 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3835a32-60a1-4ece-92b5-234e04e8ec26 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.450776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3835a32-60a1-4ece-92b5-234e04e8ec26 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.451521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3835a32-60a1-4ece-92b5-234e04e8ec26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.477526 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-f3835a32-60a1-4ece-92b5-234e04e8ec26 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:53.479023 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f3835a32-60a1-4ece-92b5-234e04e8ec26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:53.525804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04f7e6-158d-4ab5-b4bc-c90bd85f3be9 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.532094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd08075-4b46-4afb-8bde-47e453184ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.533516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd08075-4b46-4afb-8bde-47e453184ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.534002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd08075-4b46-4afb-8bde-47e453184ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.534628 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd08075-4b46-4afb-8bde-47e453184ab2 admin admin] PATH_INFO: `/v3/projects/f9807c47306440cba6a7131690dfd662/users/e3630843eaf4488a94458d50cfba9b2d/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.561827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd08075-4b46-4afb-8bde-47e453184ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9807c47306440cba6a7131690dfd662, user_id=e3630843eaf4488a94458d50cfba9b2d, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.570752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 799/1611] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:53.570752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523952d-6e1f-4e41-a98a-6c2e18be6635 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.571251 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd08075-4b46-4afb-8bde-47e453184ab2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.571458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523952d-6e1f-4e41-a98a-6c2e18be6635 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.571458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523952d-6e1f-4e41-a98a-6c2e18be6635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.599652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-b523952d-6e1f-4e41-a98a-6c2e18be6635 None None] MFA Rules not processed for user `4a4d4f493712488190cba8e9edcabf5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:53.600925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b523952d-6e1f-4e41-a98a-6c2e18be6635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:53.626728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:53.671843 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 800/1612] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:53.676673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523952d-6e1f-4e41-a98a-6c2e18be6635 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.686138 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 813/1613] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:53 2022] PUT /identity/v3/projects/f9807c47306440cba6a7131690dfd662/users/e3630843eaf4488a94458d50cfba9b2d/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:53.697123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c94a8-f119-443f-be6a-8b439c914060 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.697123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c94a8-f119-443f-be6a-8b439c914060 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.697459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c94a8-f119-443f-be6a-8b439c914060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.744767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4a3c94a8-f119-443f-be6a-8b439c914060 None None] MFA Rules not processed for user `aba1968529574fe5b4c5d114c874f428`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:53.746088 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4a3c94a8-f119-443f-be6a-8b439c914060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:53.759856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523952d-6e1f-4e41-a98a-6c2e18be6635 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.789499 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:53.831034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddfaf9-d949-4575-9c73-d71168078442 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.831034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddfaf9-d949-4575-9c73-d71168078442 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.831034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddfaf9-d949-4575-9c73-d71168078442 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.831034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddfaf9-d949-4575-9c73-d71168078442 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.853810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 814/1614] 10.222.0.78 () {58 vars in 1062 bytes} [Mon Sep 19 11:24:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:53.855889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd87dce-8ea9-485c-9c41-d127fff719c7 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.856200 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd87dce-8ea9-485c-9c41-d127fff719c7 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.863402 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd87dce-8ea9-485c-9c41-d127fff719c7 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.863402 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 815/1615] 10.222.0.78 () {54 vars in 965 bytes} [Mon Sep 19 11:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:24:53.863402 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd87dce-8ea9-485c-9c41-d127fff719c7 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.923029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddfaf9-d949-4575-9c73-d71168078442 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.930218 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddfaf9-d949-4575-9c73-d71168078442 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.933081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd87dce-8ea9-485c-9c41-d127fff719c7 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.938320 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df9755-77b4-4b2e-a105-6fadf42b3e60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.939099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df9755-77b4-4b2e-a105-6fadf42b3e60 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.939713 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df9755-77b4-4b2e-a105-6fadf42b3e60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.940316 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df9755-77b4-4b2e-a105-6fadf42b3e60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.946220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 801/1616] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:53.948685 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbae3e-d021-4e26-a828-5a13036e75c2 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.955251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbae3e-d021-4e26-a828-5a13036e75c2 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.955251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbae3e-d021-4e26-a828-5a13036e75c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.011477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-41dbae3e-d021-4e26-a828-5a13036e75c2 None None] MFA Rules not processed for user `e3630843eaf4488a94458d50cfba9b2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:54.012759 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-41dbae3e-d021-4e26-a828-5a13036e75c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:54.100288 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df9755-77b4-4b2e-a105-6fadf42b3e60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:54.102493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df9755-77b4-4b2e-a105-6fadf42b3e60 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:54.131018 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 816/1617] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:54.134468 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f5985-2877-409b-973c-4a4ca9f224e1 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.134886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f5985-2877-409b-973c-4a4ca9f224e1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.135258 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f5985-2877-409b-973c-4a4ca9f224e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.149595 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 802/1618] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:54.150852 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbae3e-d021-4e26-a828-5a13036e75c2 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:54.165239 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-158f5985-2877-409b-973c-4a4ca9f224e1 None None] MFA Rules not processed for user `aba1968529574fe5b4c5d114c874f428`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:54.165960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-158f5985-2877-409b-973c-4a4ca9f224e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:54.173898 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbae3e-d021-4e26-a828-5a13036e75c2 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:54.179923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b93606-ba05-433f-a339-aaca9f72abc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:54.180567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b93606-ba05-433f-a339-aaca9f72abc1 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.180844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b93606-ba05-433f-a339-aaca9f72abc1 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.181104 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b93606-ba05-433f-a339-aaca9f72abc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.195733 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 817/1619] 10.222.0.78 () {58 vars in 1062 bytes} [Mon Sep 19 11:24:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:54.196826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f5985-2877-409b-973c-4a4ca9f224e1 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:54.220896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f5985-2877-409b-973c-4a4ca9f224e1 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:54.228465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce919a-562b-4b98-be27-9cd797323460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:54.228465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce919a-562b-4b98-be27-9cd797323460 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.228465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce919a-562b-4b98-be27-9cd797323460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.229823 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce919a-562b-4b98-be27-9cd797323460 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.231402 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce919a-562b-4b98-be27-9cd797323460 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:54.235390 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce919a-562b-4b98-be27-9cd797323460 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:54.258394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b93606-ba05-433f-a339-aaca9f72abc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:54.261389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b93606-ba05-433f-a339-aaca9f72abc1 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:54.272452 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 803/1620] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:54.274088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b93606-ba05-433f-a339-aaca9f72abc1 service nova] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:54.289433 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b93606-ba05-433f-a339-aaca9f72abc1 service nova] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:54.297201 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb46727-5a6f-4a1c-8021-ede0290b4bb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:54.297895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb46727-5a6f-4a1c-8021-ede0290b4bb8 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.298102 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb46727-5a6f-4a1c-8021-ede0290b4bb8 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.298620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb46727-5a6f-4a1c-8021-ede0290b4bb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.397439 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 818/1621] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:54 2022] POST /identity/v3/projects => generated 365 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:54.397439 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce919a-562b-4b98-be27-9cd797323460 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:54.441554 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb46727-5a6f-4a1c-8021-ede0290b4bb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:54.445908 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb46727-5a6f-4a1c-8021-ede0290b4bb8 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:54.462972 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 804/1622] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:54.478309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce919a-562b-4b98-be27-9cd797323460 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:54.487181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbea47c-0f77-4ef8-aebd-5bbc655aa8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:54.488308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbea47c-0f77-4ef8-aebd-5bbc655aa8ee admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.488308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbea47c-0f77-4ef8-aebd-5bbc655aa8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.488496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbea47c-0f77-4ef8-aebd-5bbc655aa8ee admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.489143 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbea47c-0f77-4ef8-aebd-5bbc655aa8ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:54.497763 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbea47c-0f77-4ef8-aebd-5bbc655aa8ee admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:54.613823 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 819/1623] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:54 2022] POST /identity/v3/users => generated 289 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:24:54.618686 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb46727-5a6f-4a1c-8021-ede0290b4bb8 service nova] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:54.635592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb46727-5a6f-4a1c-8021-ede0290b4bb8 service nova] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:54.641592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4194d96-e90f-4ac9-98ec-2433db0ecf80 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.641592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4194d96-e90f-4ac9-98ec-2433db0ecf80 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.641592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4194d96-e90f-4ac9-98ec-2433db0ecf80 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.641592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 820/1624] 10.222.0.78 () {54 vars in 912 bytes} [Mon Sep 19 11:24:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:24:54.644697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89143e-f2f8-4499-8a71-0ae510eccb52 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.646019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89143e-f2f8-4499-8a71-0ae510eccb52 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.646019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89143e-f2f8-4499-8a71-0ae510eccb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.655043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85de28f-074e-49fa-9b63-27ff331a7378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:54.656882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85de28f-074e-49fa-9b63-27ff331a7378 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.656882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85de28f-074e-49fa-9b63-27ff331a7378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.657131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85de28f-074e-49fa-9b63-27ff331a7378 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.658261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85de28f-074e-49fa-9b63-27ff331a7378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:54.661107 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85de28f-074e-49fa-9b63-27ff331a7378 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:54.678544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 805/1625] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:54.685495 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-de89143e-f2f8-4499-8a71-0ae510eccb52 None None] MFA Rules not processed for user `aba1968529574fe5b4c5d114c874f428`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:54.687060 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85de28f-074e-49fa-9b63-27ff331a7378 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:54.687325 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-de89143e-f2f8-4499-8a71-0ae510eccb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:54.710614 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85de28f-074e-49fa-9b63-27ff331a7378 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:54.723901 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de69d79-ee37-414d-bdec-e6394e754352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:54.724594 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de69d79-ee37-414d-bdec-e6394e754352 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.724876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de69d79-ee37-414d-bdec-e6394e754352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.725152 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de69d79-ee37-414d-bdec-e6394e754352 admin admin] PATH_INFO: `/v3/projects/63d66661196446c9aa98d08e833b286b/users/bc89f22ac62749f0a1d85097a743307e/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.757984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:54.759608 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de69d79-ee37-414d-bdec-e6394e754352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63d66661196446c9aa98d08e833b286b, user_id=bc89f22ac62749f0a1d85097a743307e, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:54.763528 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de69d79-ee37-414d-bdec-e6394e754352 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:54.774564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 821/1626] 10.222.0.78 () {58 vars in 1009 bytes} [Mon Sep 19 11:24:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:54.789232 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89143e-f2f8-4499-8a71-0ae510eccb52 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:54.814660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:54.878304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 806/1627] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:54 2022] PUT /identity/v3/projects/63d66661196446c9aa98d08e833b286b/users/bc89f22ac62749f0a1d85097a743307e/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:54.878304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de69d79-ee37-414d-bdec-e6394e754352 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:54.886120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89143e-f2f8-4499-8a71-0ae510eccb52 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:55.026415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba486b8-7d13-480e-b5cd-9e02b43fd53f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:55.027255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba486b8-7d13-480e-b5cd-9e02b43fd53f service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:55.028174 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba486b8-7d13-480e-b5cd-9e02b43fd53f service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:55.028593 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba486b8-7d13-480e-b5cd-9e02b43fd53f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:55.036196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:55.076231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de69d79-ee37-414d-bdec-e6394e754352 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:55.087851 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e74d75-1439-45bd-aad0-764af34da619 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:55.128576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e74d75-1439-45bd-aad0-764af34da619 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:55.128576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e74d75-1439-45bd-aad0-764af34da619 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:55.128576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e74d75-1439-45bd-aad0-764af34da619 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:55.178553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba486b8-7d13-480e-b5cd-9e02b43fd53f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:55.181161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba486b8-7d13-480e-b5cd-9e02b43fd53f service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:55.211670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 822/1628] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:55.211670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba486b8-7d13-480e-b5cd-9e02b43fd53f service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:55.235304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e74d75-1439-45bd-aad0-764af34da619 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:55.238670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e74d75-1439-45bd-aad0-764af34da619 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:55.262660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 807/1629] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:55.330299 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b5021-13b6-4d5e-85f4-92371b50a4fe None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:55.330996 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b5021-13b6-4d5e-85f4-92371b50a4fe None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:55.331431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b5021-13b6-4d5e-85f4-92371b50a4fe None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:55.333297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 808/1630] 10.222.0.78 () {54 vars in 912 bytes} [Mon Sep 19 11:24:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:24:55.339147 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24601655-c970-4bea-9979-bae20886e841 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:55.712396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24601655-c970-4bea-9979-bae20886e841 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:55.712396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24601655-c970-4bea-9979-bae20886e841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:55.712396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-24601655-c970-4bea-9979-bae20886e841 None None] MFA Rules not processed for user `aba1968529574fe5b4c5d114c874f428`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:55.712396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-24601655-c970-4bea-9979-bae20886e841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:55.719447 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba486b8-7d13-480e-b5cd-9e02b43fd53f service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:55.726461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 809/1631] 10.222.0.78 () {58 vars in 1009 bytes} [Mon Sep 19 11:24:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:55.729966 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a399a-4749-4067-ac05-83f24e9a5b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:55.729966 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a399a-4749-4067-ac05-83f24e9a5b68 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:55.729966 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a399a-4749-4067-ac05-83f24e9a5b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:55.729966 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a399a-4749-4067-ac05-83f24e9a5b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:55.733158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a399a-4749-4067-ac05-83f24e9a5b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:55.733158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a399a-4749-4067-ac05-83f24e9a5b68 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:55.734347 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24601655-c970-4bea-9979-bae20886e841 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:55.734347 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24601655-c970-4bea-9979-bae20886e841 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:55.736144 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 823/1632] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:55.736578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110db05-efeb-49ee-8353-17f00b965630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:55.736578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110db05-efeb-49ee-8353-17f00b965630 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:55.736578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110db05-efeb-49ee-8353-17f00b965630 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:55.736578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110db05-efeb-49ee-8353-17f00b965630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:55.741694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a399a-4749-4067-ac05-83f24e9a5b68 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:55.741694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a399a-4749-4067-ac05-83f24e9a5b68 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:55.742273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110db05-efeb-49ee-8353-17f00b965630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:55.742273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110db05-efeb-49ee-8353-17f00b965630 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:55.742724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e5b0e-cefe-42fb-bb7a-ea798696f431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:55.742724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e5b0e-cefe-42fb-bb7a-ea798696f431 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:55.742724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e5b0e-cefe-42fb-bb7a-ea798696f431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:55.742724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e5b0e-cefe-42fb-bb7a-ea798696f431 admin admin] PATH_INFO: `/v3/projects/63d66661196446c9aa98d08e833b286b/users/bc89f22ac62749f0a1d85097a743307e/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:55.743541 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e5b0e-cefe-42fb-bb7a-ea798696f431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63d66661196446c9aa98d08e833b286b, user_id=bc89f22ac62749f0a1d85097a743307e, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:55.743541 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e5b0e-cefe-42fb-bb7a-ea798696f431 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:55.743541 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 824/1633] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:55 2022] PUT /identity/v3/projects/63d66661196446c9aa98d08e833b286b/users/bc89f22ac62749f0a1d85097a743307e/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:55.744168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 810/1634] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:24:55.744283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e0fa1-73d2-4335-87c6-a8e298e23ff4 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:55.744283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e0fa1-73d2-4335-87c6-a8e298e23ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:55.744283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e0fa1-73d2-4335-87c6-a8e298e23ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:55.744283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-580e0fa1-73d2-4335-87c6-a8e298e23ff4 None None] MFA Rules not processed for user `bc89f22ac62749f0a1d85097a743307e`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:55.744283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-580e0fa1-73d2-4335-87c6-a8e298e23ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:55.825623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 825/1635] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:56.130199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:56.846477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110db05-efeb-49ee-8353-17f00b965630 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:56.862233 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110db05-efeb-49ee-8353-17f00b965630 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:56.973586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3df557-c7b9-49cf-b881-8d1b2b777922 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:56.973586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3df557-c7b9-49cf-b881-8d1b2b777922 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:56.973586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3df557-c7b9-49cf-b881-8d1b2b777922 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:56.973586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3df557-c7b9-49cf-b881-8d1b2b777922 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:57.002164 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3df557-c7b9-49cf-b881-8d1b2b777922 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:57.003607 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3df557-c7b9-49cf-b881-8d1b2b777922 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:57.050343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 811/1636] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:57.539835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e0fa1-73d2-4335-87c6-a8e298e23ff4 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:57.556041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e0fa1-73d2-4335-87c6-a8e298e23ff4 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:57.577814 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e5fd3-52af-40db-813c-567369561ce3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:57.578679 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e5fd3-52af-40db-813c-567369561ce3 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:57.579068 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e5fd3-52af-40db-813c-567369561ce3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:57.579576 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e5fd3-52af-40db-813c-567369561ce3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:57.724247 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e5fd3-52af-40db-813c-567369561ce3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:57.725715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e5fd3-52af-40db-813c-567369561ce3 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:57.739622 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 826/1637] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:57.808023 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:59.123162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3df557-c7b9-49cf-b881-8d1b2b777922 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:59.274956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:59.307352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3df557-c7b9-49cf-b881-8d1b2b777922 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:59.337930 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af121917-5503-4515-b0cd-b87cf9e353b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:59.349288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af121917-5503-4515-b0cd-b87cf9e353b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:59.350077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af121917-5503-4515-b0cd-b87cf9e353b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:59.350580 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af121917-5503-4515-b0cd-b87cf9e353b2 admin admin] PATH_INFO: `/v3/users/42eb827ed7f440ecb60581821f686d39` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:59.357554 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af121917-5503-4515-b0cd-b87cf9e353b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42eb827ed7f440ecb60581821f686d39)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:59.361196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af121917-5503-4515-b0cd-b87cf9e353b2 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:59.632315 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-af121917-5503-4515-b0cd-b87cf9e353b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42eb827ed7f440ecb60581821f686d39'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:59.681113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-af121917-5503-4515-b0cd-b87cf9e353b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42eb827ed7f440ecb60581821f686d39'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:59.710990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-af121917-5503-4515-b0cd-b87cf9e353b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42eb827ed7f440ecb60581821f686d39'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:59.827122 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-af121917-5503-4515-b0cd-b87cf9e353b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42eb827ed7f440ecb60581821f686d39'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:59.827915 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-af121917-5503-4515-b0cd-b87cf9e353b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42eb827ed7f440ecb60581821f686d39'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:59.836278 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 812/1638] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:59 2022] DELETE /identity/v3/users/42eb827ed7f440ecb60581821f686d39 => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:59.841322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c935e70-242a-469f-9dad-00dbc5b8a537 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:59.842744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c935e70-242a-469f-9dad-00dbc5b8a537 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:59.842744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c935e70-242a-469f-9dad-00dbc5b8a537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:59.871446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-6c935e70-242a-469f-9dad-00dbc5b8a537 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:59.888311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6c935e70-242a-469f-9dad-00dbc5b8a537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:00.157504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 827/1639] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:00.261143 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af121917-5503-4515-b0cd-b87cf9e353b2 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:00.263830 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-af121917-5503-4515-b0cd-b87cf9e353b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:00.476358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af121917-5503-4515-b0cd-b87cf9e353b2 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:00.483025 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc48f0-121d-4727-a64d-713041e6eb81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:00.483679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc48f0-121d-4727-a64d-713041e6eb81 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:00.483957 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc48f0-121d-4727-a64d-713041e6eb81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:00.484222 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc48f0-121d-4727-a64d-713041e6eb81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:00.514220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc48f0-121d-4727-a64d-713041e6eb81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:00.515713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc48f0-121d-4727-a64d-713041e6eb81 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:00.569320 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 813/1640] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:01.356815 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c935e70-242a-469f-9dad-00dbc5b8a537 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:01.358744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6c935e70-242a-469f-9dad-00dbc5b8a537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:01.398749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c935e70-242a-469f-9dad-00dbc5b8a537 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:01.420103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73ffc2-8e98-4dc3-9b26-81abc29e823e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:01.420786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73ffc2-8e98-4dc3-9b26-81abc29e823e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:01.421067 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73ffc2-8e98-4dc3-9b26-81abc29e823e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:01.421341 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73ffc2-8e98-4dc3-9b26-81abc29e823e admin admin] PATH_INFO: `/v3/users/c36f918242244a37ab06617ca0ec2eab` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:01.429651 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73ffc2-8e98-4dc3-9b26-81abc29e823e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c36f918242244a37ab06617ca0ec2eab)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:01.430712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: WARNING py.warnings [None req-4f73ffc2-8e98-4dc3-9b26-81abc29e823e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:25:01.430712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: warnings.warn(msg) Sep 19 11:25:01.430712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: Sep 19 11:25:01.441016 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73ffc2-8e98-4dc3-9b26-81abc29e823e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:01.589269 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3488f4-43fa-48fe-a51a-5572bfe9e287 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:01.590967 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3488f4-43fa-48fe-a51a-5572bfe9e287 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:01.591475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3488f4-43fa-48fe-a51a-5572bfe9e287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:01.649181 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-9f3488f4-43fa-48fe-a51a-5572bfe9e287 None None] MFA Rules not processed for user `d6fee905418f4d879da460b6758d4f23`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:01.652846 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-9f3488f4-43fa-48fe-a51a-5572bfe9e287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:01.807565 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:01.811038 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 814/1641] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:01.821827 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23a8cb-8e2e-415b-bf22-1da272686d88 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:01.821827 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23a8cb-8e2e-415b-bf22-1da272686d88 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:01.822261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23a8cb-8e2e-415b-bf22-1da272686d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:01.856960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-cd23a8cb-8e2e-415b-bf22-1da272686d88 None None] MFA Rules not processed for user `d6fee905418f4d879da460b6758d4f23`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:01.858245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-cd23a8cb-8e2e-415b-bf22-1da272686d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:01.859814 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4f73ffc2-8e98-4dc3-9b26-81abc29e823e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c36f918242244a37ab06617ca0ec2eab'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:01.879979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 815/1642] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:01.890696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23a8cb-8e2e-415b-bf22-1da272686d88 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:01.897482 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-cd23a8cb-8e2e-415b-bf22-1da272686d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:01.907957 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4f73ffc2-8e98-4dc3-9b26-81abc29e823e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c36f918242244a37ab06617ca0ec2eab'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:01.934681 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4f73ffc2-8e98-4dc3-9b26-81abc29e823e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c36f918242244a37ab06617ca0ec2eab'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:02.004944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23a8cb-8e2e-415b-bf22-1da272686d88 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:02.012489 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781805e9-53d6-444c-ba30-3ccd02de1ad9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:02.013439 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781805e9-53d6-444c-ba30-3ccd02de1ad9 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:02.013898 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781805e9-53d6-444c-ba30-3ccd02de1ad9 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:02.014404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781805e9-53d6-444c-ba30-3ccd02de1ad9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:02.024854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781805e9-53d6-444c-ba30-3ccd02de1ad9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:02.026705 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781805e9-53d6-444c-ba30-3ccd02de1ad9 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:02.059249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 816/1643] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:02.111707 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4f73ffc2-8e98-4dc3-9b26-81abc29e823e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c36f918242244a37ab06617ca0ec2eab'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:02.112395 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4f73ffc2-8e98-4dc3-9b26-81abc29e823e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c36f918242244a37ab06617ca0ec2eab'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:02.122887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 828/1644] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:01 2022] DELETE /identity/v3/users/c36f918242244a37ab06617ca0ec2eab => generated 0 bytes in 767 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:02.127590 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381be17-e278-4a30-adca-fea4e8f8b7d4 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:02.127878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381be17-e278-4a30-adca-fea4e8f8b7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:02.128150 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381be17-e278-4a30-adca-fea4e8f8b7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:02.164045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-9381be17-e278-4a30-adca-fea4e8f8b7d4 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:02.165537 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-9381be17-e278-4a30-adca-fea4e8f8b7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:02.304924 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 817/1645] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:02.382731 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73ffc2-8e98-4dc3-9b26-81abc29e823e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:02.385160 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4f73ffc2-8e98-4dc3-9b26-81abc29e823e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:02.493045 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:02.597401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73ffc2-8e98-4dc3-9b26-81abc29e823e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:02.608659 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73431677-8c5f-4207-a18d-792f6fae5b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:02.609874 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73431677-8c5f-4207-a18d-792f6fae5b52 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:02.610364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73431677-8c5f-4207-a18d-792f6fae5b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:02.610848 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73431677-8c5f-4207-a18d-792f6fae5b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:02.640252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73431677-8c5f-4207-a18d-792f6fae5b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:02.642651 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73431677-8c5f-4207-a18d-792f6fae5b52 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:02.658193 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 829/1646] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:03.332544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381be17-e278-4a30-adca-fea4e8f8b7d4 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:03.333900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-9381be17-e278-4a30-adca-fea4e8f8b7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:03.360458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381be17-e278-4a30-adca-fea4e8f8b7d4 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:03.377247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526ad1c-681d-4e53-8ba9-bd1663d050f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:03.398162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526ad1c-681d-4e53-8ba9-bd1663d050f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.398449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526ad1c-681d-4e53-8ba9-bd1663d050f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.398729 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526ad1c-681d-4e53-8ba9-bd1663d050f6 admin admin] PATH_INFO: `/v3/projects/e3a8ceb9271e4036a1bba2e45e1de8ac` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.398966 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a31c16-796a-4460-b824-b43d222f0d79 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.400047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a31c16-796a-4460-b824-b43d222f0d79 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.400493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a31c16-796a-4460-b824-b43d222f0d79 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.403668 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 830/1647] 10.222.0.78 () {54 vars in 912 bytes} [Mon Sep 19 11:25:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:25:03.408147 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526ad1c-681d-4e53-8ba9-bd1663d050f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3a8ceb9271e4036a1bba2e45e1de8ac)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:03.409433 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: WARNING py.warnings [None req-f526ad1c-681d-4e53-8ba9-bd1663d050f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:25:03.409433 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: warnings.warn(msg) Sep 19 11:25:03.409433 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: Sep 19 11:25:03.415962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ddaeb-dc93-4322-a49b-1678b704bdbc None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.418069 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ddaeb-dc93-4322-a49b-1678b704bdbc None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.418703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ddaeb-dc93-4322-a49b-1678b704bdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.432280 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526ad1c-681d-4e53-8ba9-bd1663d050f6 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:03.527777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4c8ddaeb-dc93-4322-a49b-1678b704bdbc None None] MFA Rules not processed for user `dbf0049a3e3a442e945f6207477a7fd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:03.529912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4c8ddaeb-dc93-4322-a49b-1678b704bdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:03.543158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 831/1648] 10.222.0.78 () {58 vars in 1009 bytes} [Mon Sep 19 11:25:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:03.591544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ddaeb-dc93-4322-a49b-1678b704bdbc None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:03.617203 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ddaeb-dc93-4322-a49b-1678b704bdbc None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:03.689728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f526ad1c-681d-4e53-8ba9-bd1663d050f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3a8ceb9271e4036a1bba2e45e1de8ac', 'request_id': 'req-f526ad1c-681d-4e53-8ba9-bd1663d050f6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:03.732775 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f526ad1c-681d-4e53-8ba9-bd1663d050f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3a8ceb9271e4036a1bba2e45e1de8ac', 'request_id': 'req-f526ad1c-681d-4e53-8ba9-bd1663d050f6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:03.744314 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96cc33-2ff5-4a74-b6d2-2bc4ab279115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:03.745248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96cc33-2ff5-4a74-b6d2-2bc4ab279115 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.746003 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96cc33-2ff5-4a74-b6d2-2bc4ab279115 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.747241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96cc33-2ff5-4a74-b6d2-2bc4ab279115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.767605 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96cc33-2ff5-4a74-b6d2-2bc4ab279115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:03.769289 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96cc33-2ff5-4a74-b6d2-2bc4ab279115 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:03.793165 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 832/1649] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:25:03.854909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f526ad1c-681d-4e53-8ba9-bd1663d050f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3a8ceb9271e4036a1bba2e45e1de8ac', 'request_id': 'req-f526ad1c-681d-4e53-8ba9-bd1663d050f6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:03.909032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:03.923799 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f526ad1c-681d-4e53-8ba9-bd1663d050f6 admin admin] The token cache is being invalidate because project e3a8ceb9271e4036a1bba2e45e1de8ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:03.924101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f526ad1c-681d-4e53-8ba9-bd1663d050f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:03.931257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f526ad1c-681d-4e53-8ba9-bd1663d050f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:03.933221 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 818/1650] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:03 2022] DELETE /identity/v3/projects/e3a8ceb9271e4036a1bba2e45e1de8ac => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:03.938103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdf8fe-0187-4316-a32b-2853b33347e2 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.938507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdf8fe-0187-4316-a32b-2853b33347e2 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.939689 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdf8fe-0187-4316-a32b-2853b33347e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.980422 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-90cdf8fe-0187-4316-a32b-2853b33347e2 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:03.982363 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-90cdf8fe-0187-4316-a32b-2853b33347e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:04.332470 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 833/1651] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:04.333792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c760b8-e0d1-4a12-8eb8-32b1a657ea3f None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:04.333792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c760b8-e0d1-4a12-8eb8-32b1a657ea3f None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:04.333792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c760b8-e0d1-4a12-8eb8-32b1a657ea3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:04.333792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-a3c760b8-e0d1-4a12-8eb8-32b1a657ea3f None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:04.333792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a3c760b8-e0d1-4a12-8eb8-32b1a657ea3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:04.333792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 819/1652] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:04.335038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cdf8fe-0187-4316-a32b-2853b33347e2 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:04.335038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cdf8fe-0187-4316-a32b-2853b33347e2 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:04.335038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23266f-9e16-47c8-84f0-24d9e68592cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:04.335038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23266f-9e16-47c8-84f0-24d9e68592cb admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:04.336758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23266f-9e16-47c8-84f0-24d9e68592cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:04.336758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23266f-9e16-47c8-84f0-24d9e68592cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:04.336758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23266f-9e16-47c8-84f0-24d9e68592cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:04.336758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23266f-9e16-47c8-84f0-24d9e68592cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:04.336758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23266f-9e16-47c8-84f0-24d9e68592cb admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:04.336758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 834/1653] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:25:04.338172 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c760b8-e0d1-4a12-8eb8-32b1a657ea3f None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:04.338172 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c760b8-e0d1-4a12-8eb8-32b1a657ea3f None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:04.347448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54395c-fe8f-411b-a67b-ade94433c5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:04.347634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54395c-fe8f-411b-a67b-ade94433c5c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:04.347850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54395c-fe8f-411b-a67b-ade94433c5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:04.348132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54395c-fe8f-411b-a67b-ade94433c5c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:04.348872 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54395c-fe8f-411b-a67b-ade94433c5c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:04.352210 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54395c-fe8f-411b-a67b-ade94433c5c1 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:04.466484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23266f-9e16-47c8-84f0-24d9e68592cb admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:04.469167 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ce23266f-9e16-47c8-84f0-24d9e68592cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:04.497709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23266f-9e16-47c8-84f0-24d9e68592cb admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:04.527211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a05b2-aa43-4399-8b16-3aa7b741fe51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:04.527211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a05b2-aa43-4399-8b16-3aa7b741fe51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:04.527211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a05b2-aa43-4399-8b16-3aa7b741fe51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:04.527211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a05b2-aa43-4399-8b16-3aa7b741fe51 admin admin] PATH_INFO: `/v3/projects/68f214f15a7b46f2a05c1d8ff0ea6692` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:04.528213 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a05b2-aa43-4399-8b16-3aa7b741fe51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68f214f15a7b46f2a05c1d8ff0ea6692)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:04.528293 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 820/1654] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:04 2022] POST /identity/v3/projects => generated 373 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:04.530895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54395c-fe8f-411b-a67b-ade94433c5c1 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:04.533475 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a05b2-aa43-4399-8b16-3aa7b741fe51 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:04.714624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54395c-fe8f-411b-a67b-ade94433c5c1 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:04.724377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca5148-81bc-44b3-b273-d4e509632423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:04.724795 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca5148-81bc-44b3-b273-d4e509632423 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:04.725176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca5148-81bc-44b3-b273-d4e509632423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:04.726455 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca5148-81bc-44b3-b273-d4e509632423 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:04.726455 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca5148-81bc-44b3-b273-d4e509632423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:04.731078 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca5148-81bc-44b3-b273-d4e509632423 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:04.816115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:04.841192 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f82a05b2-aa43-4399-8b16-3aa7b741fe51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68f214f15a7b46f2a05c1d8ff0ea6692', 'request_id': 'req-f82a05b2-aa43-4399-8b16-3aa7b741fe51'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:04.896361 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 821/1655] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:04 2022] POST /identity/v3/users => generated 287 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:04.901123 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca5148-81bc-44b3-b273-d4e509632423 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:04.924234 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca5148-81bc-44b3-b273-d4e509632423 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:04.949995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f82a05b2-aa43-4399-8b16-3aa7b741fe51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68f214f15a7b46f2a05c1d8ff0ea6692', 'request_id': 'req-f82a05b2-aa43-4399-8b16-3aa7b741fe51'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:04.963933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f82a05b2-aa43-4399-8b16-3aa7b741fe51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68f214f15a7b46f2a05c1d8ff0ea6692', 'request_id': 'req-f82a05b2-aa43-4399-8b16-3aa7b741fe51'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:05.031769 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f82a05b2-aa43-4399-8b16-3aa7b741fe51 admin admin] The token cache is being invalidate because project 68f214f15a7b46f2a05c1d8ff0ea6692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:05.038629 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f82a05b2-aa43-4399-8b16-3aa7b741fe51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:05.038629 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f82a05b2-aa43-4399-8b16-3aa7b741fe51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:05.048436 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 835/1656] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:04 2022] DELETE /identity/v3/projects/68f214f15a7b46f2a05c1d8ff0ea6692 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:05.052417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fb15d-05b6-4e92-8613-af3046df9294 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:05.055082 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fb15d-05b6-4e92-8613-af3046df9294 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:05.056620 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fb15d-05b6-4e92-8613-af3046df9294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:05.124834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126ea36-9ea9-47e6-af5e-cc17ba682a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:05.125459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126ea36-9ea9-47e6-af5e-cc17ba682a7c admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:05.125735 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126ea36-9ea9-47e6-af5e-cc17ba682a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:05.126007 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126ea36-9ea9-47e6-af5e-cc17ba682a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:05.127888 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1126ea36-9ea9-47e6-af5e-cc17ba682a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:05.138925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1126ea36-9ea9-47e6-af5e-cc17ba682a7c admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:05.155343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ee5fb15d-05b6-4e92-8613-af3046df9294 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:05.157930 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ee5fb15d-05b6-4e92-8613-af3046df9294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:05.169214 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 822/1657] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:05.175929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126ea36-9ea9-47e6-af5e-cc17ba682a7c admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:05.180386 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1126ea36-9ea9-47e6-af5e-cc17ba682a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:05.191234 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 836/1658] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:05.191234 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fb15d-05b6-4e92-8613-af3046df9294 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:05.196827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ee5fb15d-05b6-4e92-8613-af3046df9294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:05.223054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fb15d-05b6-4e92-8613-af3046df9294 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:05.239959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f8d59-8f5a-45cd-aca8-9187e080f806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:05.240601 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f8d59-8f5a-45cd-aca8-9187e080f806 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:05.240882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f8d59-8f5a-45cd-aca8-9187e080f806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:05.241156 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f8d59-8f5a-45cd-aca8-9187e080f806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:05.241893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f8d59-8f5a-45cd-aca8-9187e080f806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:05.245584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f8d59-8f5a-45cd-aca8-9187e080f806 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:05.281531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 837/1659] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:05.287692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6b118-d582-4627-b545-07a2bff04448 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:05.291507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6b118-d582-4627-b545-07a2bff04448 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:05.291921 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6b118-d582-4627-b545-07a2bff04448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:05.329850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d0d6b118-d582-4627-b545-07a2bff04448 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:05.332496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d0d6b118-d582-4627-b545-07a2bff04448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:05.362128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 838/1660] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:05.363742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6b118-d582-4627-b545-07a2bff04448 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:05.387803 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6b118-d582-4627-b545-07a2bff04448 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:05.398167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126ea36-9ea9-47e6-af5e-cc17ba682a7c admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:05.398759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc549a7-8b78-4736-b81f-bf4a9f4c0921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:05.398924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc549a7-8b78-4736-b81f-bf4a9f4c0921 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:05.401662 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc549a7-8b78-4736-b81f-bf4a9f4c0921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:05.402635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc549a7-8b78-4736-b81f-bf4a9f4c0921 admin admin] PATH_INFO: `/v3/projects/fdda1f1942f64c4d85e682e03a9cb967/users/4445136db42a4173a51006e9859f134a/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:05.425923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb1553-dacb-497c-9a71-c96f33cdd46d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:05.426576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb1553-dacb-497c-9a71-c96f33cdd46d service placement] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:05.426854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb1553-dacb-497c-9a71-c96f33cdd46d service placement] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:05.427131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb1553-dacb-497c-9a71-c96f33cdd46d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:05.440761 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc549a7-8b78-4736-b81f-bf4a9f4c0921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdda1f1942f64c4d85e682e03a9cb967, user_id=4445136db42a4173a51006e9859f134a, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:05.442212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-bddb1553-dacb-497c-9a71-c96f33cdd46d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:05.443715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc549a7-8b78-4736-b81f-bf4a9f4c0921 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:05.472449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb1553-dacb-497c-9a71-c96f33cdd46d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:05.474435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb1553-dacb-497c-9a71-c96f33cdd46d service placement] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:05.504555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 823/1661] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:05.506344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b9303-026b-4d08-bba9-cb6a92d17117 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:05.506797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b9303-026b-4d08-bba9-cb6a92d17117 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:05.507279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b9303-026b-4d08-bba9-cb6a92d17117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:05.530916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-290b9303-026b-4d08-bba9-cb6a92d17117 None None] MFA Rules not processed for user `d6fee905418f4d879da460b6758d4f23`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:05.532570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-290b9303-026b-4d08-bba9-cb6a92d17117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:05.635092 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 839/1662] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:05 2022] PUT /identity/v3/projects/fdda1f1942f64c4d85e682e03a9cb967/users/4445136db42a4173a51006e9859f134a/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:05.641664 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc549a7-8b78-4736-b81f-bf4a9f4c0921 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:05.660787 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 824/1663] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:05.660787 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a015f-460a-47ce-921b-5a784c68a78b None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:05.661879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a015f-460a-47ce-921b-5a784c68a78b None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:05.662900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a015f-460a-47ce-921b-5a784c68a78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:05.750034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-8a6a015f-460a-47ce-921b-5a784c68a78b None None] MFA Rules not processed for user `eec3ef0842b948deb7ecc46f7c56dd04`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:05.759731 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-8a6a015f-460a-47ce-921b-5a784c68a78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:05.849832 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc549a7-8b78-4736-b81f-bf4a9f4c0921 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:05.873771 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197a468-ad2c-4105-89b9-ee38e7ce1c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:05.874408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197a468-ad2c-4105-89b9-ee38e7ce1c3d admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:05.874688 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197a468-ad2c-4105-89b9-ee38e7ce1c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:05.874962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197a468-ad2c-4105-89b9-ee38e7ce1c3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:05.888257 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197a468-ad2c-4105-89b9-ee38e7ce1c3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:05.888584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197a468-ad2c-4105-89b9-ee38e7ce1c3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:05.890951 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197a468-ad2c-4105-89b9-ee38e7ce1c3d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:05.916626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 840/1664] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:25:05.918264 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ebc4d-796a-464d-9df4-89d2f9ef6e17 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:05.918555 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ebc4d-796a-464d-9df4-89d2f9ef6e17 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:05.918846 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ebc4d-796a-464d-9df4-89d2f9ef6e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:06.026165 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-664ebc4d-796a-464d-9df4-89d2f9ef6e17 None None] MFA Rules not processed for user `4445136db42a4173a51006e9859f134a`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:06.027810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-664ebc4d-796a-464d-9df4-89d2f9ef6e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:06.027911 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 825/1665] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:06.028823 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a015f-460a-47ce-921b-5a784c68a78b None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:06.050652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a015f-460a-47ce-921b-5a784c68a78b None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:06.059905 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fec20-893b-427b-9002-da5afff5805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:06.061396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fec20-893b-427b-9002-da5afff5805d admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:06.062653 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fec20-893b-427b-9002-da5afff5805d admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:06.063055 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fec20-893b-427b-9002-da5afff5805d admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:06.063967 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fec20-893b-427b-9002-da5afff5805d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:06.066182 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fec20-893b-427b-9002-da5afff5805d admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:06.285897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 826/1666] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:06 2022] POST /identity/v3/projects => generated 373 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:06.286093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 841/1667] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:06.288846 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ebc4d-796a-464d-9df4-89d2f9ef6e17 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:06.290799 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ed0ca-70e6-407a-a185-1d78513719ef None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:06.291214 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ed0ca-70e6-407a-a185-1d78513719ef None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:06.291506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ed0ca-70e6-407a-a185-1d78513719ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:06.333610 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ebc4d-796a-464d-9df4-89d2f9ef6e17 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:06.362841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-660ed0ca-70e6-407a-a185-1d78513719ef None None] MFA Rules not processed for user `eec3ef0842b948deb7ecc46f7c56dd04`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:06.367120 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-660ed0ca-70e6-407a-a185-1d78513719ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:06.523729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9598d0-5b86-4f2e-a745-1c3d1f181992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:06.524366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9598d0-5b86-4f2e-a745-1c3d1f181992 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:06.524705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9598d0-5b86-4f2e-a745-1c3d1f181992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:06.529893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9598d0-5b86-4f2e-a745-1c3d1f181992 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:06.530641 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9598d0-5b86-4f2e-a745-1c3d1f181992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:06.540103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9598d0-5b86-4f2e-a745-1c3d1f181992 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:06.606910 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 827/1668] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:06.614776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ed0ca-70e6-407a-a185-1d78513719ef None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:06.650572 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ed0ca-70e6-407a-a185-1d78513719ef None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:06.669087 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d504914-e85f-4da3-a152-787108f7f4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:06.671518 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d504914-e85f-4da3-a152-787108f7f4d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:06.671803 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d504914-e85f-4da3-a152-787108f7f4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:06.672079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d504914-e85f-4da3-a152-787108f7f4d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:06.676573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d504914-e85f-4da3-a152-787108f7f4d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:06.686220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d504914-e85f-4da3-a152-787108f7f4d2 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:06.764910 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 842/1669] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:06 2022] POST /identity/v3/users => generated 287 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:06.772659 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9598d0-5b86-4f2e-a745-1c3d1f181992 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:06.782993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5a9598d0-5b86-4f2e-a745-1c3d1f181992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:06.912757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 828/1670] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:06 2022] POST /identity/v3/projects => generated 373 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:06.914690 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377b983-15bf-49c1-9841-9604c08ac65f None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:06.915161 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377b983-15bf-49c1-9841-9604c08ac65f None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:06.916514 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377b983-15bf-49c1-9841-9604c08ac65f None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:06.924182 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 829/1671] 10.222.0.78 () {54 vars in 912 bytes} [Mon Sep 19 11:25:06 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:25:06.925123 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377b983-15bf-49c1-9841-9604c08ac65f None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:06.942615 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9598d0-5b86-4f2e-a745-1c3d1f181992 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:07.065961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:07.095456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e378a6-4c8d-4abc-a438-24ac0702617e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:07.096531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e378a6-4c8d-4abc-a438-24ac0702617e service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:07.096827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e378a6-4c8d-4abc-a438-24ac0702617e service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:07.097287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e378a6-4c8d-4abc-a438-24ac0702617e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:07.142851 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377b983-15bf-49c1-9841-9604c08ac65f None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:07.152674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a86c5c-9842-49c5-9bd2-92d90fddee3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:07.153325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a86c5c-9842-49c5-9bd2-92d90fddee3e admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:07.153604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a86c5c-9842-49c5-9bd2-92d90fddee3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:07.154440 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a86c5c-9842-49c5-9bd2-92d90fddee3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:07.155205 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a86c5c-9842-49c5-9bd2-92d90fddee3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:07.157877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a86c5c-9842-49c5-9bd2-92d90fddee3e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:07.173662 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 830/1672] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:07.174575 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a86c5c-9842-49c5-9bd2-92d90fddee3e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:07.190645 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a86c5c-9842-49c5-9bd2-92d90fddee3e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:07.196626 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7d8e5-9103-452f-b931-28cd2ac3f988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:07.197280 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7d8e5-9103-452f-b931-28cd2ac3f988 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:07.197558 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7d8e5-9103-452f-b931-28cd2ac3f988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:07.197831 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7d8e5-9103-452f-b931-28cd2ac3f988 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:07.198979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7d8e5-9103-452f-b931-28cd2ac3f988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:07.201302 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7d8e5-9103-452f-b931-28cd2ac3f988 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:07.230515 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e378a6-4c8d-4abc-a438-24ac0702617e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:07.232637 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e378a6-4c8d-4abc-a438-24ac0702617e service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:07.243951 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 843/1673] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:07.245370 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ec22e-ed0a-419d-99c9-db5f13fa653f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:07.246942 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ec22e-ed0a-419d-99c9-db5f13fa653f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:07.247436 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ec22e-ed0a-419d-99c9-db5f13fa653f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:07.278692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-793ec22e-ed0a-419d-99c9-db5f13fa653f None None] MFA Rules not processed for user `dbf0049a3e3a442e945f6207477a7fd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:07.280432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-793ec22e-ed0a-419d-99c9-db5f13fa653f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:07.336251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:07.379548 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 831/1674] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:07 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:07.380801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103bd28-a48c-4503-ab5b-0dbc1c904681 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:07.381083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103bd28-a48c-4503-ab5b-0dbc1c904681 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:07.381423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103bd28-a48c-4503-ab5b-0dbc1c904681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:07.423725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 844/1675] 10.222.0.78 () {58 vars in 1009 bytes} [Mon Sep 19 11:25:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:07.424952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ec22e-ed0a-419d-99c9-db5f13fa653f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:07.428411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-7103bd28-a48c-4503-ab5b-0dbc1c904681 None None] MFA Rules not processed for user `405b3189506a44bc93555c40917b4cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:07.431979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7103bd28-a48c-4503-ab5b-0dbc1c904681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:07.439549 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ec22e-ed0a-419d-99c9-db5f13fa653f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:07.452736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a4f4d-4806-414e-8298-d5889934b82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:07.453731 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a4f4d-4806-414e-8298-d5889934b82b admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:07.454971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a4f4d-4806-414e-8298-d5889934b82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:07.455489 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a4f4d-4806-414e-8298-d5889934b82b admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:07.456648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a4f4d-4806-414e-8298-d5889934b82b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:07.458517 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a4f4d-4806-414e-8298-d5889934b82b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:07.486547 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 845/1676] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:07.488680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a4f4d-4806-414e-8298-d5889934b82b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:07.514045 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a4f4d-4806-414e-8298-d5889934b82b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:07.553492 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd988d59-e695-4c5e-bd00-e6a9da3fe979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:07.554539 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd988d59-e695-4c5e-bd00-e6a9da3fe979 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:07.555100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd988d59-e695-4c5e-bd00-e6a9da3fe979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:07.555651 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd988d59-e695-4c5e-bd00-e6a9da3fe979 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:07.557813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd988d59-e695-4c5e-bd00-e6a9da3fe979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:07.560930 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd988d59-e695-4c5e-bd00-e6a9da3fe979 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:07.576183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 832/1677] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:07.577428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103bd28-a48c-4503-ab5b-0dbc1c904681 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:07.582585 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 846/1678] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:07.586344 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd988d59-e695-4c5e-bd00-e6a9da3fe979 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:07.589072 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fd988d59-e695-4c5e-bd00-e6a9da3fe979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:07.626417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd988d59-e695-4c5e-bd00-e6a9da3fe979 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:07.641820 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103bd28-a48c-4503-ab5b-0dbc1c904681 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:07.645423 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f113f6-3202-40a4-bbbc-bba579a185c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:07.645423 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f113f6-3202-40a4-bbbc-bba579a185c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:07.645423 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f113f6-3202-40a4-bbbc-bba579a185c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:07.645423 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f113f6-3202-40a4-bbbc-bba579a185c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:07.673149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a04095-08a1-4cd8-812b-d4b36bad6ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:07.673149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a04095-08a1-4cd8-812b-d4b36bad6ae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:07.673149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a04095-08a1-4cd8-812b-d4b36bad6ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:07.673531 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a04095-08a1-4cd8-812b-d4b36bad6ae2 admin admin] PATH_INFO: `/v3/projects/2fe22248259d41beb1e7018ea86e5d9a/users/22a236ae660741019d8e47f4e4e64e41/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:07.683468 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f113f6-3202-40a4-bbbc-bba579a185c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:07.684932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f113f6-3202-40a4-bbbc-bba579a185c4 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:07.720806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 847/1679] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:07.720806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f113f6-3202-40a4-bbbc-bba579a185c4 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:07.727630 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a04095-08a1-4cd8-812b-d4b36bad6ae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fe22248259d41beb1e7018ea86e5d9a, user_id=22a236ae660741019d8e47f4e4e64e41, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:07.734862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a04095-08a1-4cd8-812b-d4b36bad6ae2 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:07.756339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f113f6-3202-40a4-bbbc-bba579a185c4 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:07.774652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88c6ac-4fe8-485f-b289-6446efe4d807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:07.775382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88c6ac-4fe8-485f-b289-6446efe4d807 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:07.775669 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88c6ac-4fe8-485f-b289-6446efe4d807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:07.775953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88c6ac-4fe8-485f-b289-6446efe4d807 admin admin] PATH_INFO: `/v3/projects/cd701640afa142ee8fc7b3bf09a35420/users/39a2143eb79b46ad8ddb8912f75d6966/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:07.804923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88c6ac-4fe8-485f-b289-6446efe4d807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd701640afa142ee8fc7b3bf09a35420, user_id=39a2143eb79b46ad8ddb8912f75d6966, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:07.811101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88c6ac-4fe8-485f-b289-6446efe4d807 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:07.885119 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 833/1680] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:07 2022] PUT /identity/v3/projects/2fe22248259d41beb1e7018ea86e5d9a/users/22a236ae660741019d8e47f4e4e64e41/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:07.898347 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3068da-b1a4-461d-b6f2-793916b17ec2 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:07.898665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3068da-b1a4-461d-b6f2-793916b17ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:07.898934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3068da-b1a4-461d-b6f2-793916b17ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:07.939041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 848/1681] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:07 2022] PUT /identity/v3/projects/cd701640afa142ee8fc7b3bf09a35420/users/39a2143eb79b46ad8ddb8912f75d6966/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:07.940100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88c6ac-4fe8-485f-b289-6446efe4d807 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:07.947409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-8f3068da-b1a4-461d-b6f2-793916b17ec2 None None] MFA Rules not processed for user `22a236ae660741019d8e47f4e4e64e41`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:07.952008 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-8f3068da-b1a4-461d-b6f2-793916b17ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:08.088081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 834/1682] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:08.094688 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3068da-b1a4-461d-b6f2-793916b17ec2 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:08.099333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88c6ac-4fe8-485f-b289-6446efe4d807 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:08.114004 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41413741-3ee5-47fc-890f-5038f4c9f6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:08.114650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41413741-3ee5-47fc-890f-5038f4c9f6de admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:08.114932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41413741-3ee5-47fc-890f-5038f4c9f6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:08.115224 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41413741-3ee5-47fc-890f-5038f4c9f6de admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:08.117492 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41413741-3ee5-47fc-890f-5038f4c9f6de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:08.120096 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41413741-3ee5-47fc-890f-5038f4c9f6de admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:08.121752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3068da-b1a4-461d-b6f2-793916b17ec2 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:08.134378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf74d5-7cc5-436d-8130-92fcb05812b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:08.135005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf74d5-7cc5-436d-8130-92fcb05812b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:08.135336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf74d5-7cc5-436d-8130-92fcb05812b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:08.135604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf74d5-7cc5-436d-8130-92fcb05812b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:08.136835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf74d5-7cc5-436d-8130-92fcb05812b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:08.138800 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf74d5-7cc5-436d-8130-92fcb05812b0 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:08.144000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 849/1683] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:08.147616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41413741-3ee5-47fc-890f-5038f4c9f6de admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:08.175384 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41413741-3ee5-47fc-890f-5038f4c9f6de admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:08.183932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1bba9-e655-4cdb-93ad-5f25d2cd835b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:08.186467 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1bba9-e655-4cdb-93ad-5f25d2cd835b admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:08.186758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1bba9-e655-4cdb-93ad-5f25d2cd835b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:08.187045 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1bba9-e655-4cdb-93ad-5f25d2cd835b admin admin] PATH_INFO: `/v3/projects/cd701640afa142ee8fc7b3bf09a35420/users/39a2143eb79b46ad8ddb8912f75d6966/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:08.189466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1bba9-e655-4cdb-93ad-5f25d2cd835b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd701640afa142ee8fc7b3bf09a35420, user_id=39a2143eb79b46ad8ddb8912f75d6966, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:08.192514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1bba9-e655-4cdb-93ad-5f25d2cd835b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:08.209680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 850/1684] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:08 2022] PUT /identity/v3/projects/cd701640afa142ee8fc7b3bf09a35420/users/39a2143eb79b46ad8ddb8912f75d6966/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:25:08.220309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27ef47-c932-442b-a50c-4f5e5156d5fd None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:08.220549 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27ef47-c932-442b-a50c-4f5e5156d5fd None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:08.220825 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27ef47-c932-442b-a50c-4f5e5156d5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:08.271684 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4b27ef47-c932-442b-a50c-4f5e5156d5fd None None] MFA Rules not processed for user `39a2143eb79b46ad8ddb8912f75d6966`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:08.273603 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4b27ef47-c932-442b-a50c-4f5e5156d5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:08.337864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 835/1685] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:08 2022] POST /identity/v3/projects => generated 373 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:08.338812 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf74d5-7cc5-436d-8130-92fcb05812b0 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:08.442629 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 851/1686] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:08.456049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27ef47-c932-442b-a50c-4f5e5156d5fd None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:08.456796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4b27ef47-c932-442b-a50c-4f5e5156d5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:08.526941 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf74d5-7cc5-436d-8130-92fcb05812b0 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:08.532653 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319fc34-1181-4dfc-a6c2-84356350c803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:08.533373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319fc34-1181-4dfc-a6c2-84356350c803 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:08.533723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319fc34-1181-4dfc-a6c2-84356350c803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:08.534056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319fc34-1181-4dfc-a6c2-84356350c803 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:08.534704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319fc34-1181-4dfc-a6c2-84356350c803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:08.536679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319fc34-1181-4dfc-a6c2-84356350c803 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:08.546196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27ef47-c932-442b-a50c-4f5e5156d5fd None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:08.600107 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8e3e5-9c18-4de2-8646-bf18cd413fc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:08.600601 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8e3e5-9c18-4de2-8646-bf18cd413fc7 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:08.600914 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8e3e5-9c18-4de2-8646-bf18cd413fc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:08.601186 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8e3e5-9c18-4de2-8646-bf18cd413fc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:08.659290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 836/1687] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:08 2022] POST /identity/v3/users => generated 293 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:08.672043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319fc34-1181-4dfc-a6c2-84356350c803 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:08.699123 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319fc34-1181-4dfc-a6c2-84356350c803 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:08.702137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebae005-4537-477e-ad43-92b49b3d9d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:08.703091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebae005-4537-477e-ad43-92b49b3d9d9d admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:08.703531 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebae005-4537-477e-ad43-92b49b3d9d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:08.703878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebae005-4537-477e-ad43-92b49b3d9d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:08.704648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebae005-4537-477e-ad43-92b49b3d9d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:08.706462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebae005-4537-477e-ad43-92b49b3d9d9d admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:08.737714 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 837/1688] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:08.738902 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebae005-4537-477e-ad43-92b49b3d9d9d admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:08.755434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebae005-4537-477e-ad43-92b49b3d9d9d admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:08.757322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8e3e5-9c18-4de2-8646-bf18cd413fc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:08.758795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8e3e5-9c18-4de2-8646-bf18cd413fc7 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:08.763687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e3555-0ef3-4401-a4be-072fb38c5ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:08.765721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e3555-0ef3-4401-a4be-072fb38c5ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:08.765721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e3555-0ef3-4401-a4be-072fb38c5ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:08.765721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e3555-0ef3-4401-a4be-072fb38c5ffe admin admin] PATH_INFO: `/v3/projects/4b157b74e6144c3f9c10a8016bf684f9/users/5f8e7973285b4583bb4fe3039ba49636/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:08.785779 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 852/1689] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:25:08.788048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e3555-0ef3-4401-a4be-072fb38c5ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b157b74e6144c3f9c10a8016bf684f9, user_id=5f8e7973285b4583bb4fe3039ba49636, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:08.793107 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e3555-0ef3-4401-a4be-072fb38c5ffe admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:08.843339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8e3e5-9c18-4de2-8646-bf18cd413fc7 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:08.845396 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-19f8e3e5-9c18-4de2-8646-bf18cd413fc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:08.886268 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 838/1690] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:08 2022] PUT /identity/v3/projects/4b157b74e6144c3f9c10a8016bf684f9/users/5f8e7973285b4583bb4fe3039ba49636/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:08.887895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e3555-0ef3-4401-a4be-072fb38c5ffe admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:09.028304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8e3e5-9c18-4de2-8646-bf18cd413fc7 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:09.121610 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e3555-0ef3-4401-a4be-072fb38c5ffe admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:09.128060 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddccf97-5d13-4f1a-adf4-5b4725df8dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:09.135044 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddccf97-5d13-4f1a-adf4-5b4725df8dec admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:09.135836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddccf97-5d13-4f1a-adf4-5b4725df8dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:09.136324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddccf97-5d13-4f1a-adf4-5b4725df8dec admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:09.137640 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddccf97-5d13-4f1a-adf4-5b4725df8dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:09.154110 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddccf97-5d13-4f1a-adf4-5b4725df8dec admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:09.219626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71edcc3-764c-46ac-b9cb-bbee3e56e2c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:09.226067 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71edcc3-764c-46ac-b9cb-bbee3e56e2c4 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:09.226294 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71edcc3-764c-46ac-b9cb-bbee3e56e2c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:09.226570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71edcc3-764c-46ac-b9cb-bbee3e56e2c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:09.231260 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 839/1691] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:09.232880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddccf97-5d13-4f1a-adf4-5b4725df8dec admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:09.259375 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddccf97-5d13-4f1a-adf4-5b4725df8dec admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:09.270551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136a59c-dfa9-4424-bad0-f4762a097cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:09.271337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136a59c-dfa9-4424-bad0-f4762a097cc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:09.271564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136a59c-dfa9-4424-bad0-f4762a097cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:09.271906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136a59c-dfa9-4424-bad0-f4762a097cc0 admin admin] PATH_INFO: `/v3/projects/4b157b74e6144c3f9c10a8016bf684f9/users/5f8e7973285b4583bb4fe3039ba49636/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:09.287246 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136a59c-dfa9-4424-bad0-f4762a097cc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b157b74e6144c3f9c10a8016bf684f9, user_id=5f8e7973285b4583bb4fe3039ba49636, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:09.289420 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136a59c-dfa9-4424-bad0-f4762a097cc0 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:09.320798 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 840/1692] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:09 2022] PUT /identity/v3/projects/4b157b74e6144c3f9c10a8016bf684f9/users/5f8e7973285b4583bb4fe3039ba49636/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:09.330451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423f68e-2ecc-46c7-9648-99da5844eb68 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:09.330736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423f68e-2ecc-46c7-9648-99da5844eb68 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:09.331013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423f68e-2ecc-46c7-9648-99da5844eb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:09.339126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71edcc3-764c-46ac-b9cb-bbee3e56e2c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:09.342946 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71edcc3-764c-46ac-b9cb-bbee3e56e2c4 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:09.371061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 853/1693] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:09.402533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:09.412923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-a423f68e-2ecc-46c7-9648-99da5844eb68 None None] MFA Rules not processed for user `5f8e7973285b4583bb4fe3039ba49636`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:09.412923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a423f68e-2ecc-46c7-9648-99da5844eb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:09.565019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 841/1694] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:09.723913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71edcc3-764c-46ac-b9cb-bbee3e56e2c4 service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:09.725555 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a71edcc3-764c-46ac-b9cb-bbee3e56e2c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:09.788922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423f68e-2ecc-46c7-9648-99da5844eb68 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:09.788922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a423f68e-2ecc-46c7-9648-99da5844eb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:09.814909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71edcc3-764c-46ac-b9cb-bbee3e56e2c4 service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:09.882261 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf8c64-10db-4526-aa70-095a05fa68aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:09.882814 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf8c64-10db-4526-aa70-095a05fa68aa service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:09.883151 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf8c64-10db-4526-aa70-095a05fa68aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:09.883530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf8c64-10db-4526-aa70-095a05fa68aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:09.895147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cf8c64-10db-4526-aa70-095a05fa68aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:09.897142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cf8c64-10db-4526-aa70-095a05fa68aa service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:09.909702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423f68e-2ecc-46c7-9648-99da5844eb68 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:09.919031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 854/1695] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:09.922625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f2654-62c1-47ff-bbff-e86723fab5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:09.923329 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f2654-62c1-47ff-bbff-e86723fab5b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:09.923612 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f2654-62c1-47ff-bbff-e86723fab5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:09.924058 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f2654-62c1-47ff-bbff-e86723fab5b5 admin admin] PATH_INFO: `/v3/users/4a4d4f493712488190cba8e9edcabf5b` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:09.926212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f2654-62c1-47ff-bbff-e86723fab5b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a4d4f493712488190cba8e9edcabf5b)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:09.929155 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f2654-62c1-47ff-bbff-e86723fab5b5 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:10.371493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:10.392025 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-b71f2654-62c1-47ff-bbff-e86723fab5b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a4d4f493712488190cba8e9edcabf5b'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:10.432395 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-b71f2654-62c1-47ff-bbff-e86723fab5b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a4d4f493712488190cba8e9edcabf5b'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:10.484071 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-b71f2654-62c1-47ff-bbff-e86723fab5b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a4d4f493712488190cba8e9edcabf5b'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:10.555431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf8c64-10db-4526-aa70-095a05fa68aa service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:10.566614 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf8c64-10db-4526-aa70-095a05fa68aa service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:10.573059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f89d2-d8cc-406e-b12d-5c19d1e0714b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:10.574068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-b71f2654-62c1-47ff-bbff-e86723fab5b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a4d4f493712488190cba8e9edcabf5b'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:10.574230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f89d2-d8cc-406e-b12d-5c19d1e0714b service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:10.574805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f89d2-d8cc-406e-b12d-5c19d1e0714b service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:10.578366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f89d2-d8cc-406e-b12d-5c19d1e0714b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:10.579257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-b71f2654-62c1-47ff-bbff-e86723fab5b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a4d4f493712488190cba8e9edcabf5b'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:10.592952 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 842/1696] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:09 2022] DELETE /identity/v3/users/4a4d4f493712488190cba8e9edcabf5b => generated 0 bytes in 808 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:10.606882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ecb95-0350-4986-850e-66758cc8ae3c None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:10.608414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ecb95-0350-4986-850e-66758cc8ae3c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:10.609052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ecb95-0350-4986-850e-66758cc8ae3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:10.677303 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-7a2ecb95-0350-4986-850e-66758cc8ae3c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:10.683373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7a2ecb95-0350-4986-850e-66758cc8ae3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:10.699382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f89d2-d8cc-406e-b12d-5c19d1e0714b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:10.704470 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f89d2-d8cc-406e-b12d-5c19d1e0714b service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:10.738827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 855/1697] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:25:10.870287 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 843/1698] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:10.881455 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f89d2-d8cc-406e-b12d-5c19d1e0714b service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:11.025450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f89d2-d8cc-406e-b12d-5c19d1e0714b service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:11.103852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a9260-e635-4b72-8c2b-a8893ac84452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:11.104699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a9260-e635-4b72-8c2b-a8893ac84452 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:11.105096 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a9260-e635-4b72-8c2b-a8893ac84452 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:11.105704 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a9260-e635-4b72-8c2b-a8893ac84452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:11.128364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a9260-e635-4b72-8c2b-a8893ac84452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:11.129926 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a9260-e635-4b72-8c2b-a8893ac84452 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:11.149635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 856/1699] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:11.768767 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:12.013190 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ecb95-0350-4986-850e-66758cc8ae3c None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:12.015793 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7a2ecb95-0350-4986-850e-66758cc8ae3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:12.034492 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ecb95-0350-4986-850e-66758cc8ae3c None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:12.040515 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e3957-4dd7-4629-b376-0dd985fb758f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:12.041407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e3957-4dd7-4629-b376-0dd985fb758f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:12.041809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e3957-4dd7-4629-b376-0dd985fb758f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:12.042223 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e3957-4dd7-4629-b376-0dd985fb758f admin admin] PATH_INFO: `/v3/users/4445136db42a4173a51006e9859f134a` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:12.043608 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e3957-4dd7-4629-b376-0dd985fb758f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4445136db42a4173a51006e9859f134a)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:12.045886 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e3957-4dd7-4629-b376-0dd985fb758f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:12.206163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a9260-e635-4b72-8c2b-a8893ac84452 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:12.208008 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-025a9260-e635-4b72-8c2b-a8893ac84452 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:12.228109 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a9260-e635-4b72-8c2b-a8893ac84452 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:12.244899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34dd048-fdd5-4944-8b9e-d32e099f7e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:12.251267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dd048-fdd5-4944-8b9e-d32e099f7e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:12.251267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dd048-fdd5-4944-8b9e-d32e099f7e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:12.251267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dd048-fdd5-4944-8b9e-d32e099f7e50 admin admin] PATH_INFO: `/v3/projects/0f6f09e176314205bbad48d0d0a382b3` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:12.253707 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34dd048-fdd5-4944-8b9e-d32e099f7e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f6f09e176314205bbad48d0d0a382b3)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:12.256489 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34dd048-fdd5-4944-8b9e-d32e099f7e50 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:12.380688 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1e0e3957-4dd7-4629-b376-0dd985fb758f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4445136db42a4173a51006e9859f134a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:12.410001 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1e0e3957-4dd7-4629-b376-0dd985fb758f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4445136db42a4173a51006e9859f134a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:12.424871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1e0e3957-4dd7-4629-b376-0dd985fb758f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4445136db42a4173a51006e9859f134a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:12.506157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1e0e3957-4dd7-4629-b376-0dd985fb758f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4445136db42a4173a51006e9859f134a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:12.506938 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1e0e3957-4dd7-4629-b376-0dd985fb758f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4445136db42a4173a51006e9859f134a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:12.516234 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 844/1700] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:12 2022] DELETE /identity/v3/users/4445136db42a4173a51006e9859f134a => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:12.522468 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd1b74-0950-4993-bb83-8cbf09a231ca None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:12.522771 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd1b74-0950-4993-bb83-8cbf09a231ca None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:12.523081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd1b74-0950-4993-bb83-8cbf09a231ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:12.530810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a34dd048-fdd5-4944-8b9e-d32e099f7e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f6f09e176314205bbad48d0d0a382b3', 'request_id': 'req-a34dd048-fdd5-4944-8b9e-d32e099f7e50'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:12.558404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-1dcd1b74-0950-4993-bb83-8cbf09a231ca None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:12.560691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1dcd1b74-0950-4993-bb83-8cbf09a231ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:12.679951 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a34dd048-fdd5-4944-8b9e-d32e099f7e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f6f09e176314205bbad48d0d0a382b3', 'request_id': 'req-a34dd048-fdd5-4944-8b9e-d32e099f7e50'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:12.691698 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a34dd048-fdd5-4944-8b9e-d32e099f7e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f6f09e176314205bbad48d0d0a382b3', 'request_id': 'req-a34dd048-fdd5-4944-8b9e-d32e099f7e50'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:12.710458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 845/1701] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:12.738291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a34dd048-fdd5-4944-8b9e-d32e099f7e50 admin admin] The token cache is being invalidate because project 0f6f09e176314205bbad48d0d0a382b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:12.738609 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a34dd048-fdd5-4944-8b9e-d32e099f7e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:12.740915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:12.753925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a34dd048-fdd5-4944-8b9e-d32e099f7e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:12.756034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 857/1702] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:12 2022] DELETE /identity/v3/projects/0f6f09e176314205bbad48d0d0a382b3 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:25:12.761446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4449692-53ef-48c7-872b-bd7272a4013f None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:12.762054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4449692-53ef-48c7-872b-bd7272a4013f None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:12.762481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4449692-53ef-48c7-872b-bd7272a4013f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:12.764930 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34dd048-fdd5-4944-8b9e-d32e099f7e50 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:12.766930 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a34dd048-fdd5-4944-8b9e-d32e099f7e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:12.865449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-b4449692-53ef-48c7-872b-bd7272a4013f None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:12.868703 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b4449692-53ef-48c7-872b-bd7272a4013f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:12.916600 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 846/1703] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:12.922559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f88d2-0bb9-4cb6-a3be-bcdd1254028b None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:12.924139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f88d2-0bb9-4cb6-a3be-bcdd1254028b None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:12.925351 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f88d2-0bb9-4cb6-a3be-bcdd1254028b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:12.991018 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34dd048-fdd5-4944-8b9e-d32e099f7e50 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:13.100864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-699f88d2-0bb9-4cb6-a3be-bcdd1254028b None None] MFA Rules not processed for user `5a667fa07b1148998959835d9995ad60`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:13.102123 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-699f88d2-0bb9-4cb6-a3be-bcdd1254028b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:13.169504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244a0e6-5856-4c34-a5fd-991dcff65b28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:13.170411 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244a0e6-5856-4c34-a5fd-991dcff65b28 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:13.170679 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244a0e6-5856-4c34-a5fd-991dcff65b28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:13.170959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244a0e6-5856-4c34-a5fd-991dcff65b28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:13.178907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0244a0e6-5856-4c34-a5fd-991dcff65b28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:13.219576 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244a0e6-5856-4c34-a5fd-991dcff65b28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:13.221037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244a0e6-5856-4c34-a5fd-991dcff65b28 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:13.252776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 858/1704] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:13.262590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f45fd9-a849-4f9c-84c2-dd4381397eda None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:13.262877 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f45fd9-a849-4f9c-84c2-dd4381397eda None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:13.266290 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f45fd9-a849-4f9c-84c2-dd4381397eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:13.318981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 847/1705] 10.222.0.78 () {58 vars in 1009 bytes} [Mon Sep 19 11:25:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:13.353028 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f4f45fd9-a849-4f9c-84c2-dd4381397eda None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:13.357295 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f4f45fd9-a849-4f9c-84c2-dd4381397eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:13.390440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 859/1706] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:13.395889 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f88d2-0bb9-4cb6-a3be-bcdd1254028b None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:13.416480 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f88d2-0bb9-4cb6-a3be-bcdd1254028b None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:13.432268 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e190d79-1eef-4cf3-af99-a28785456307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:13.434619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e190d79-1eef-4cf3-af99-a28785456307 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:13.434917 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e190d79-1eef-4cf3-af99-a28785456307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:13.435217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e190d79-1eef-4cf3-af99-a28785456307 admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:13.439368 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e190d79-1eef-4cf3-af99-a28785456307 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:13.439717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e190d79-1eef-4cf3-af99-a28785456307 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:13.444859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e190d79-1eef-4cf3-af99-a28785456307 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:13.485711 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 848/1707] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:25:13.486606 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f45fd9-a849-4f9c-84c2-dd4381397eda None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:13.502366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f45fd9-a849-4f9c-84c2-dd4381397eda None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:13.517767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f495213-d9bc-4c81-a0f0-3a2c2dcded1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:13.518290 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f495213-d9bc-4c81-a0f0-3a2c2dcded1a admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:13.518574 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f495213-d9bc-4c81-a0f0-3a2c2dcded1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:13.518853 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f495213-d9bc-4c81-a0f0-3a2c2dcded1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:13.522656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f495213-d9bc-4c81-a0f0-3a2c2dcded1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:13.525147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f495213-d9bc-4c81-a0f0-3a2c2dcded1a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:13.702733 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 860/1708] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:13 2022] POST /identity/v3/projects => generated 365 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:13.707062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e190d79-1eef-4cf3-af99-a28785456307 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:13.718642 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e190d79-1eef-4cf3-af99-a28785456307 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:13.834664 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f495213-d9bc-4c81-a0f0-3a2c2dcded1a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:13.837851 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9f495213-d9bc-4c81-a0f0-3a2c2dcded1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:13.877557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0780dd9-acb2-47f4-bb70-01701f6bbed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:13.878079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0780dd9-acb2-47f4-bb70-01701f6bbed7 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:13.878448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0780dd9-acb2-47f4-bb70-01701f6bbed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:13.878720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0780dd9-acb2-47f4-bb70-01701f6bbed7 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:13.879399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0780dd9-acb2-47f4-bb70-01701f6bbed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:13.882103 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0780dd9-acb2-47f4-bb70-01701f6bbed7 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:14.000212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 849/1709] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:13 2022] POST /identity/v3/users => generated 283 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:14.003666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0780dd9-acb2-47f4-bb70-01701f6bbed7 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:14.038654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0780dd9-acb2-47f4-bb70-01701f6bbed7 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:14.054837 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8adec2-8880-4245-bd1a-65694dd3bafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:14.056998 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8adec2-8880-4245-bd1a-65694dd3bafd admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:14.077955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8adec2-8880-4245-bd1a-65694dd3bafd admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:14.079781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8adec2-8880-4245-bd1a-65694dd3bafd admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:14.081741 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8adec2-8880-4245-bd1a-65694dd3bafd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:14.086353 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8adec2-8880-4245-bd1a-65694dd3bafd admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:14.125895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 850/1710] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:25:14.129872 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8adec2-8880-4245-bd1a-65694dd3bafd admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:14.143946 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f495213-d9bc-4c81-a0f0-3a2c2dcded1a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:14.148596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8adec2-8880-4245-bd1a-65694dd3bafd admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:14.154121 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2758bb2-b1f5-4e27-b35c-aa2bdccd9e42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf462b2aa3fc4e2ca58851549cf0c644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:14.154851 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2758bb2-b1f5-4e27-b35c-aa2bdccd9e42 service swift] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:14.155133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2758bb2-b1f5-4e27-b35c-aa2bdccd9e42 service swift] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:14.173199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2758bb2-b1f5-4e27-b35c-aa2bdccd9e42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:14.173618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade495a8-3e85-4727-b718-bd59afe59f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:14.174721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade495a8-3e85-4727-b718-bd59afe59f60 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:14.175059 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade495a8-3e85-4727-b718-bd59afe59f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:14.177299 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade495a8-3e85-4727-b718-bd59afe59f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:14.179161 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade495a8-3e85-4727-b718-bd59afe59f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:14.199589 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade495a8-3e85-4727-b718-bd59afe59f60 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:14.257023 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 851/1711] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:25:14.257023 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77395e91-1927-4c37-ba40-fcf63fc60703 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:14.259104 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77395e91-1927-4c37-ba40-fcf63fc60703 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:14.259471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77395e91-1927-4c37-ba40-fcf63fc60703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:14.310959 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-77395e91-1927-4c37-ba40-fcf63fc60703 None None] MFA Rules not processed for user `dbf0049a3e3a442e945f6207477a7fd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:14.327225 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-77395e91-1927-4c37-ba40-fcf63fc60703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:14.386867 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2758bb2-b1f5-4e27-b35c-aa2bdccd9e42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:14.389269 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2758bb2-b1f5-4e27-b35c-aa2bdccd9e42 service swift] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:14.419448 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 861/1712] 10.222.0.78 () {60 vars in 1384 bytes} [Mon Sep 19 11:25:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:25:14.422513 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2758bb2-b1f5-4e27-b35c-aa2bdccd9e42 service swift] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:14.430650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:14.485522 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2758bb2-b1f5-4e27-b35c-aa2bdccd9e42 service swift] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:14.497062 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52613343-678f-4ac7-83ee-dfd86274a8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:14.500840 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52613343-678f-4ac7-83ee-dfd86274a8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:14.502347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52613343-678f-4ac7-83ee-dfd86274a8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:14.502913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52613343-678f-4ac7-83ee-dfd86274a8b2 admin admin] PATH_INFO: `/v3/projects/59bbe0b3d779464db5b6e996d4a4febe/users/3e2eb1d3dd62412d8569c41d141e01cd/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:14.515708 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 852/1713] 10.222.0.83 () {58 vars in 1015 bytes} [Mon Sep 19 11:25:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:14.548208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52613343-678f-4ac7-83ee-dfd86274a8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bbe0b3d779464db5b6e996d4a4febe, user_id=3e2eb1d3dd62412d8569c41d141e01cd, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:14.559415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52613343-678f-4ac7-83ee-dfd86274a8b2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:14.577075 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77395e91-1927-4c37-ba40-fcf63fc60703 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:14.594845 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77395e91-1927-4c37-ba40-fcf63fc60703 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:14.607151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a9a32-54c7-46ca-b94a-60d791aaa933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:14.611909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a9a32-54c7-46ca-b94a-60d791aaa933 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:14.611909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a9a32-54c7-46ca-b94a-60d791aaa933 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:14.611909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a9a32-54c7-46ca-b94a-60d791aaa933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:14.646445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a9a32-54c7-46ca-b94a-60d791aaa933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:14.646445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a9a32-54c7-46ca-b94a-60d791aaa933 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:14.693881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 862/1714] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:14 2022] PUT /identity/v3/projects/59bbe0b3d779464db5b6e996d4a4febe/users/3e2eb1d3dd62412d8569c41d141e01cd/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:14.698412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1307d81-c137-4010-9d3c-4fee3fb30a41 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:14.706562 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1307d81-c137-4010-9d3c-4fee3fb30a41 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:14.707233 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1307d81-c137-4010-9d3c-4fee3fb30a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:14.714810 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 853/1715] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:14.716074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a9a32-54c7-46ca-b94a-60d791aaa933 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:14.724919 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-6e7a9a32-54c7-46ca-b94a-60d791aaa933 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:14.806643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a9a32-54c7-46ca-b94a-60d791aaa933 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:14.816025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e1307d81-c137-4010-9d3c-4fee3fb30a41 None None] MFA Rules not processed for user `3e2eb1d3dd62412d8569c41d141e01cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:14.830728 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e1307d81-c137-4010-9d3c-4fee3fb30a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:15.000890 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 863/1716] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:15.011503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1307d81-c137-4010-9d3c-4fee3fb30a41 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:15.056069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910db65-2548-43ec-a91e-625fef266b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:15.059929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910db65-2548-43ec-a91e-625fef266b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:15.060620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910db65-2548-43ec-a91e-625fef266b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:15.062735 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910db65-2548-43ec-a91e-625fef266b3d admin admin] PATH_INFO: `/v3/projects/fdda1f1942f64c4d85e682e03a9cb967` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:15.071051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910db65-2548-43ec-a91e-625fef266b3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdda1f1942f64c4d85e682e03a9cb967)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:15.096614 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910db65-2548-43ec-a91e-625fef266b3d admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:15.302775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1307d81-c137-4010-9d3c-4fee3fb30a41 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:15.320736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf87753-9694-4a17-adda-0f77b4d07028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:15.321657 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf87753-9694-4a17-adda-0f77b4d07028 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:15.322096 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf87753-9694-4a17-adda-0f77b4d07028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:15.322526 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf87753-9694-4a17-adda-0f77b4d07028 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:15.324065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf87753-9694-4a17-adda-0f77b4d07028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:15.326454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf87753-9694-4a17-adda-0f77b4d07028 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:15.398063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-d910db65-2548-43ec-a91e-625fef266b3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdda1f1942f64c4d85e682e03a9cb967', 'request_id': 'req-d910db65-2548-43ec-a91e-625fef266b3d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:15.422517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-d910db65-2548-43ec-a91e-625fef266b3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdda1f1942f64c4d85e682e03a9cb967', 'request_id': 'req-d910db65-2548-43ec-a91e-625fef266b3d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:15.496952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 864/1717] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:15 2022] POST /identity/v3/projects => generated 363 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:15.502076 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf87753-9694-4a17-adda-0f77b4d07028 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:15.534376 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-d910db65-2548-43ec-a91e-625fef266b3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdda1f1942f64c4d85e682e03a9cb967', 'request_id': 'req-d910db65-2548-43ec-a91e-625fef266b3d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:15.553672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-d910db65-2548-43ec-a91e-625fef266b3d admin admin] The token cache is being invalidate because project fdda1f1942f64c4d85e682e03a9cb967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:15.554141 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-d910db65-2548-43ec-a91e-625fef266b3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:15.555439 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-d910db65-2548-43ec-a91e-625fef266b3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:15.558948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 854/1718] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:14 2022] DELETE /identity/v3/projects/fdda1f1942f64c4d85e682e03a9cb967 => generated 0 bytes in 844 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:15.563447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910db65-2548-43ec-a91e-625fef266b3d admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:15.566461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d910db65-2548-43ec-a91e-625fef266b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:15.650535 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf87753-9694-4a17-adda-0f77b4d07028 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:15.665016 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89fc3b-2975-46c0-90e8-4c0bb0a43e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:15.665652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89fc3b-2975-46c0-90e8-4c0bb0a43e42 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:15.665935 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89fc3b-2975-46c0-90e8-4c0bb0a43e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:15.666211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89fc3b-2975-46c0-90e8-4c0bb0a43e42 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:15.667069 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89fc3b-2975-46c0-90e8-4c0bb0a43e42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:15.669042 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89fc3b-2975-46c0-90e8-4c0bb0a43e42 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:15.701278 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:15.784534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910db65-2548-43ec-a91e-625fef266b3d admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:15.802900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72208de4-59aa-4acc-8b77-fa2c1e87659f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:15.808142 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72208de4-59aa-4acc-8b77-fa2c1e87659f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:15.808524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72208de4-59aa-4acc-8b77-fa2c1e87659f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:15.808804 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72208de4-59aa-4acc-8b77-fa2c1e87659f admin admin] PATH_INFO: `/v3/users/39a2143eb79b46ad8ddb8912f75d6966` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:15.811061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72208de4-59aa-4acc-8b77-fa2c1e87659f admin admin] RBAC: Authorizing `identity:delete_user(user_id=39a2143eb79b46ad8ddb8912f75d6966)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:15.815523 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72208de4-59aa-4acc-8b77-fa2c1e87659f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:15.866539 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 865/1719] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:15 2022] POST /identity/v3/users => generated 288 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:15.867771 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89fc3b-2975-46c0-90e8-4c0bb0a43e42 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:15.883552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8c89fc3b-2975-46c0-90e8-4c0bb0a43e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:15.927307 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89fc3b-2975-46c0-90e8-4c0bb0a43e42 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:15.937840 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c8b00-f12b-4271-81a5-276831710163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:15.939337 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c8b00-f12b-4271-81a5-276831710163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:15.941560 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c8b00-f12b-4271-81a5-276831710163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:15.942093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c8b00-f12b-4271-81a5-276831710163 admin admin] PATH_INFO: `/v3/users/07c3aeab01d2454eb2b8c781b93f645d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:15.947627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c8b00-f12b-4271-81a5-276831710163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07c3aeab01d2454eb2b8c781b93f645d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:15.953703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c8b00-f12b-4271-81a5-276831710163 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:16.156049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-72208de4-59aa-4acc-8b77-fa2c1e87659f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39a2143eb79b46ad8ddb8912f75d6966'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:16.199616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-72208de4-59aa-4acc-8b77-fa2c1e87659f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39a2143eb79b46ad8ddb8912f75d6966'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:16.237926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-72208de4-59aa-4acc-8b77-fa2c1e87659f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39a2143eb79b46ad8ddb8912f75d6966'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:16.353318 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3e7c8b00-f12b-4271-81a5-276831710163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07c3aeab01d2454eb2b8c781b93f645d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:16.358632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-72208de4-59aa-4acc-8b77-fa2c1e87659f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39a2143eb79b46ad8ddb8912f75d6966'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:16.358632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-72208de4-59aa-4acc-8b77-fa2c1e87659f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39a2143eb79b46ad8ddb8912f75d6966'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:16.375862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 855/1720] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:15 2022] DELETE /identity/v3/users/39a2143eb79b46ad8ddb8912f75d6966 => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:25:16.375862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72208de4-59aa-4acc-8b77-fa2c1e87659f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:16.376448 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3e7c8b00-f12b-4271-81a5-276831710163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07c3aeab01d2454eb2b8c781b93f645d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:16.378414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-72208de4-59aa-4acc-8b77-fa2c1e87659f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:16.386579 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3e7c8b00-f12b-4271-81a5-276831710163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07c3aeab01d2454eb2b8c781b93f645d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:16.523083 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3e7c8b00-f12b-4271-81a5-276831710163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07c3aeab01d2454eb2b8c781b93f645d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:16.525423 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3e7c8b00-f12b-4271-81a5-276831710163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07c3aeab01d2454eb2b8c781b93f645d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:16.537294 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 866/1721] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:15 2022] DELETE /identity/v3/users/07c3aeab01d2454eb2b8c781b93f645d => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:16.539454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c8b00-f12b-4271-81a5-276831710163 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:16.546103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3e7c8b00-f12b-4271-81a5-276831710163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:16.572137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72208de4-59aa-4acc-8b77-fa2c1e87659f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:16.661137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675e4cf-eb62-475d-b1ed-309eb8e63a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:16.662408 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675e4cf-eb62-475d-b1ed-309eb8e63a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:16.662408 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675e4cf-eb62-475d-b1ed-309eb8e63a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:16.662771 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675e4cf-eb62-475d-b1ed-309eb8e63a09 admin admin] PATH_INFO: `/v3/users/e3630843eaf4488a94458d50cfba9b2d` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:16.664578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675e4cf-eb62-475d-b1ed-309eb8e63a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3630843eaf4488a94458d50cfba9b2d)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:16.669137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675e4cf-eb62-475d-b1ed-309eb8e63a09 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:16.700485 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c8b00-f12b-4271-81a5-276831710163 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:16.728133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd0163-f8b6-41ec-821f-abbeb9c75fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:16.728133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd0163-f8b6-41ec-821f-abbeb9c75fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:16.728133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd0163-f8b6-41ec-821f-abbeb9c75fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:16.728133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd0163-f8b6-41ec-821f-abbeb9c75fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:16.728759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd0163-f8b6-41ec-821f-abbeb9c75fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:16.728759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd0163-f8b6-41ec-821f-abbeb9c75fe8 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:16.740062 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 867/1722] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:16.744144 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43347db3-abe8-4dbb-824e-43e84db00805 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:16.744144 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43347db3-abe8-4dbb-824e-43e84db00805 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:16.744144 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43347db3-abe8-4dbb-824e-43e84db00805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:16.774230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-43347db3-abe8-4dbb-824e-43e84db00805 None None] MFA Rules not processed for user `405b3189506a44bc93555c40917b4cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:16.776132 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-43347db3-abe8-4dbb-824e-43e84db00805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:16.898441 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 868/1723] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:16.899898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15170f37-9dc9-44d5-9e91-ab0540de9c5f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:16.900176 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15170f37-9dc9-44d5-9e91-ab0540de9c5f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:16.900788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15170f37-9dc9-44d5-9e91-ab0540de9c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:16.959769 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-15170f37-9dc9-44d5-9e91-ab0540de9c5f None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:16.967392 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-15170f37-9dc9-44d5-9e91-ab0540de9c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:17.015281 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 869/1724] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:17.016910 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15170f37-9dc9-44d5-9e91-ab0540de9c5f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:17.058750 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15170f37-9dc9-44d5-9e91-ab0540de9c5f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:17.071673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db5286-0a96-4db9-a0a7-ea620187d43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:17.072381 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db5286-0a96-4db9-a0a7-ea620187d43a admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:17.072665 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db5286-0a96-4db9-a0a7-ea620187d43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:17.072948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db5286-0a96-4db9-a0a7-ea620187d43a admin admin] PATH_INFO: `/v3/projects/141b57f0e32442f7b234c1f41f612c5b/users/d5438ba7ea0e463dac23348fcf55ca6c/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:17.122034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db5286-0a96-4db9-a0a7-ea620187d43a admin admin] RBAC: Authorizing `identity:create_grant(project_id=141b57f0e32442f7b234c1f41f612c5b, user_id=d5438ba7ea0e463dac23348fcf55ca6c, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:17.125093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db5286-0a96-4db9-a0a7-ea620187d43a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:17.186896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:17.250032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-9675e4cf-eb62-475d-b1ed-309eb8e63a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3630843eaf4488a94458d50cfba9b2d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:17.267379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 870/1725] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:17 2022] PUT /identity/v3/projects/141b57f0e32442f7b234c1f41f612c5b/users/d5438ba7ea0e463dac23348fcf55ca6c/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:17.273297 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db5286-0a96-4db9-a0a7-ea620187d43a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:17.281762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-51db5286-0a96-4db9-a0a7-ea620187d43a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:17.311737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-9675e4cf-eb62-475d-b1ed-309eb8e63a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3630843eaf4488a94458d50cfba9b2d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:17.329928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-9675e4cf-eb62-475d-b1ed-309eb8e63a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3630843eaf4488a94458d50cfba9b2d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:17.389970 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db5286-0a96-4db9-a0a7-ea620187d43a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:17.394929 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0538893-3623-4f82-8725-adcd5be1c794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:17.395973 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0538893-3623-4f82-8725-adcd5be1c794 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:17.396467 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0538893-3623-4f82-8725-adcd5be1c794 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:17.396900 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0538893-3623-4f82-8725-adcd5be1c794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:17.407566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-9675e4cf-eb62-475d-b1ed-309eb8e63a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3630843eaf4488a94458d50cfba9b2d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:17.408556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-9675e4cf-eb62-475d-b1ed-309eb8e63a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3630843eaf4488a94458d50cfba9b2d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:17.419882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 856/1726] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:16 2022] DELETE /identity/v3/users/e3630843eaf4488a94458d50cfba9b2d => generated 0 bytes in 1042 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:17.422123 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675e4cf-eb62-475d-b1ed-309eb8e63a09 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:17.424764 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-9675e4cf-eb62-475d-b1ed-309eb8e63a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:17.535376 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0538893-3623-4f82-8725-adcd5be1c794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:17.538255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0538893-3623-4f82-8725-adcd5be1c794 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:17.560043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 871/1727] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:17.562229 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074622b7-0846-44c9-a854-eb033119ab7c None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:17.563326 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074622b7-0846-44c9-a854-eb033119ab7c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:17.567208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074622b7-0846-44c9-a854-eb033119ab7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:17.567464 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675e4cf-eb62-475d-b1ed-309eb8e63a09 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:17.578621 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc5890-5ecf-4c15-9cfd-17b419cea671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:17.579677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc5890-5ecf-4c15-9cfd-17b419cea671 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:17.579810 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc5890-5ecf-4c15-9cfd-17b419cea671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:17.580130 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc5890-5ecf-4c15-9cfd-17b419cea671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:17.580822 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc5890-5ecf-4c15-9cfd-17b419cea671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:17.586521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc5890-5ecf-4c15-9cfd-17b419cea671 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:17.606674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 857/1728] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:17.610713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc5890-5ecf-4c15-9cfd-17b419cea671 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:17.620886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-074622b7-0846-44c9-a854-eb033119ab7c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:17.624611 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc5890-5ecf-4c15-9cfd-17b419cea671 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:17.624911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-074622b7-0846-44c9-a854-eb033119ab7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:17.631385 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b40ba9-622a-4bf6-a199-19a4de5b14c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:17.632258 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b40ba9-622a-4bf6-a199-19a4de5b14c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:17.632643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b40ba9-622a-4bf6-a199-19a4de5b14c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:17.633116 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b40ba9-622a-4bf6-a199-19a4de5b14c7 admin admin] PATH_INFO: `/v3/projects/141b57f0e32442f7b234c1f41f612c5b/users/d5438ba7ea0e463dac23348fcf55ca6c/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:17.636049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b40ba9-622a-4bf6-a199-19a4de5b14c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=141b57f0e32442f7b234c1f41f612c5b, user_id=d5438ba7ea0e463dac23348fcf55ca6c, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:17.640293 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b40ba9-622a-4bf6-a199-19a4de5b14c7 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:17.651543 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 872/1729] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:17.655337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 858/1730] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:17 2022] PUT /identity/v3/projects/141b57f0e32442f7b234c1f41f612c5b/users/d5438ba7ea0e463dac23348fcf55ca6c/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:17.669687 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e5137-f82a-4af4-ab97-5c5c0168837f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:17.670104 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e5137-f82a-4af4-ab97-5c5c0168837f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:17.670262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e5137-f82a-4af4-ab97-5c5c0168837f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:17.674184 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b40ba9-622a-4bf6-a199-19a4de5b14c7 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:17.677824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e1b40ba9-622a-4bf6-a199-19a4de5b14c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:17.738159 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-291e5137-f82a-4af4-ab97-5c5c0168837f None None] MFA Rules not processed for user `d5438ba7ea0e463dac23348fcf55ca6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:17.739771 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-291e5137-f82a-4af4-ab97-5c5c0168837f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:17.771356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b40ba9-622a-4bf6-a199-19a4de5b14c7 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:17.773577 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc23e7-e4c0-4d44-95ca-bc3067b5ab26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:17.774645 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc23e7-e4c0-4d44-95ca-bc3067b5ab26 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:17.775751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc23e7-e4c0-4d44-95ca-bc3067b5ab26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:17.776470 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc23e7-e4c0-4d44-95ca-bc3067b5ab26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:17.879263 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc23e7-e4c0-4d44-95ca-bc3067b5ab26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:17.882438 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc23e7-e4c0-4d44-95ca-bc3067b5ab26 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:17.898365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 859/1731] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:17.965280 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 873/1732] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:17 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:17.978669 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc23e7-e4c0-4d44-95ca-bc3067b5ab26 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:17.983226 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-9ecc23e7-e4c0-4d44-95ca-bc3067b5ab26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:18.002281 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:18.146104 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc23e7-e4c0-4d44-95ca-bc3067b5ab26 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:18.152567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2daa82c-5219-46df-b557-83ca72fb006d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:18.162697 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2daa82c-5219-46df-b557-83ca72fb006d service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:18.172267 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2daa82c-5219-46df-b557-83ca72fb006d service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:18.174064 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2daa82c-5219-46df-b557-83ca72fb006d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:18.218280 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2daa82c-5219-46df-b557-83ca72fb006d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:18.222829 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2daa82c-5219-46df-b557-83ca72fb006d service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:18.257383 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 860/1733] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:18.428086 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e5137-f82a-4af4-ab97-5c5c0168837f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:18.429894 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-291e5137-f82a-4af4-ab97-5c5c0168837f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:18.470636 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e5137-f82a-4af4-ab97-5c5c0168837f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:18.477935 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1b25d-6448-4cf9-a899-fafdf59d80a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:18.478644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1b25d-6448-4cf9-a899-fafdf59d80a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:18.478931 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1b25d-6448-4cf9-a899-fafdf59d80a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:18.479986 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1b25d-6448-4cf9-a899-fafdf59d80a7 admin admin] PATH_INFO: `/v3/projects/cd701640afa142ee8fc7b3bf09a35420` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:18.483714 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1b25d-6448-4cf9-a899-fafdf59d80a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd701640afa142ee8fc7b3bf09a35420)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:18.485792 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1b25d-6448-4cf9-a899-fafdf59d80a7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:18.701759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7fa1b25d-6448-4cf9-a899-fafdf59d80a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd701640afa142ee8fc7b3bf09a35420', 'request_id': 'req-7fa1b25d-6448-4cf9-a899-fafdf59d80a7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:18.905619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7fa1b25d-6448-4cf9-a899-fafdf59d80a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd701640afa142ee8fc7b3bf09a35420', 'request_id': 'req-7fa1b25d-6448-4cf9-a899-fafdf59d80a7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:18.919606 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7fa1b25d-6448-4cf9-a899-fafdf59d80a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd701640afa142ee8fc7b3bf09a35420', 'request_id': 'req-7fa1b25d-6448-4cf9-a899-fafdf59d80a7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:18.972100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7fa1b25d-6448-4cf9-a899-fafdf59d80a7 admin admin] The token cache is being invalidate because project cd701640afa142ee8fc7b3bf09a35420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:18.976446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7fa1b25d-6448-4cf9-a899-fafdf59d80a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:18.984418 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7fa1b25d-6448-4cf9-a899-fafdf59d80a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:18.992278 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 874/1734] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:18 2022] DELETE /identity/v3/projects/cd701640afa142ee8fc7b3bf09a35420 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:18.999404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2949d9-ad53-4bf1-b0d5-17c8f253022d None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:19.000237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2949d9-ad53-4bf1-b0d5-17c8f253022d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:19.001159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2949d9-ad53-4bf1-b0d5-17c8f253022d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:19.054438 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:19.080433 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-7d2949d9-ad53-4bf1-b0d5-17c8f253022d None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:19.081692 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7d2949d9-ad53-4bf1-b0d5-17c8f253022d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:19.288698 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 861/1735] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:19.296895 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf69694-c837-4ef6-b6e1-b1e21ff925dc None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:19.297180 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf69694-c837-4ef6-b6e1-b1e21ff925dc None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:19.301766 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf69694-c837-4ef6-b6e1-b1e21ff925dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:19.360635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-eaf69694-c837-4ef6-b6e1-b1e21ff925dc None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:19.370935 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-eaf69694-c837-4ef6-b6e1-b1e21ff925dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:19.381313 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 875/1736] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:19.385827 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2949d9-ad53-4bf1-b0d5-17c8f253022d None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:19.396574 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2949d9-ad53-4bf1-b0d5-17c8f253022d None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:19.402352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184f984-2bdc-424b-b85b-af267ba46eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:19.402993 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184f984-2bdc-424b-b85b-af267ba46eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:19.403319 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184f984-2bdc-424b-b85b-af267ba46eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:19.403598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184f984-2bdc-424b-b85b-af267ba46eb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:19.404272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184f984-2bdc-424b-b85b-af267ba46eb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:19.404578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184f984-2bdc-424b-b85b-af267ba46eb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:19.406282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184f984-2bdc-424b-b85b-af267ba46eb1 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:19.428176 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0f1ad-bd0c-4e57-96e5-5fa65621a8a0 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:19.428480 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0f1ad-bd0c-4e57-96e5-5fa65621a8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:19.428760 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0f1ad-bd0c-4e57-96e5-5fa65621a8a0 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:19.430608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 876/1737] 10.222.0.78 () {54 vars in 921 bytes} [Mon Sep 19 11:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:25:19.433834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 862/1738] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:25:19.434382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0f1ad-bd0c-4e57-96e5-5fa65621a8a0 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:19.438975 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f42a6-37a2-4aae-93b0-c54073402802 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:19.446882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f42a6-37a2-4aae-93b0-c54073402802 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:19.446882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f42a6-37a2-4aae-93b0-c54073402802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:19.459999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0f1ad-bd0c-4e57-96e5-5fa65621a8a0 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:19.473391 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63bd783-fd43-450f-b010-bf043a95757a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:19.474046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bd783-fd43-450f-b010-bf043a95757a admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:19.474354 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bd783-fd43-450f-b010-bf043a95757a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:19.474627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bd783-fd43-450f-b010-bf043a95757a admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:19.475351 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63bd783-fd43-450f-b010-bf043a95757a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:19.477269 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63bd783-fd43-450f-b010-bf043a95757a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:19.492237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-691f42a6-37a2-4aae-93b0-c54073402802 None None] MFA Rules not processed for user `aba1968529574fe5b4c5d114c874f428`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:19.493503 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-691f42a6-37a2-4aae-93b0-c54073402802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:19.584883 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 877/1739] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:19 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:25:19.585911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63bd783-fd43-450f-b010-bf043a95757a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:19.649596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 863/1740] 10.222.0.78 () {58 vars in 1018 bytes} [Mon Sep 19 11:25:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:19.673430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63bd783-fd43-450f-b010-bf043a95757a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:19.681600 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8e7cc-6689-41c3-86d7-99d2c883166a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:19.683055 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8e7cc-6689-41c3-86d7-99d2c883166a admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:19.683518 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8e7cc-6689-41c3-86d7-99d2c883166a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:19.683911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8e7cc-6689-41c3-86d7-99d2c883166a admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:19.685099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8e7cc-6689-41c3-86d7-99d2c883166a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:19.687366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8e7cc-6689-41c3-86d7-99d2c883166a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:19.863137 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 878/1741] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:19 2022] POST /identity/v3/users => generated 288 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:19.864466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f42a6-37a2-4aae-93b0-c54073402802 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:19.876107 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f42a6-37a2-4aae-93b0-c54073402802 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:19.880862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398a40c-b9e9-4a50-9b71-99a7d0318385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:19.881475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398a40c-b9e9-4a50-9b71-99a7d0318385 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:19.881756 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398a40c-b9e9-4a50-9b71-99a7d0318385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:19.882031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398a40c-b9e9-4a50-9b71-99a7d0318385 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:19.882679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398a40c-b9e9-4a50-9b71-99a7d0318385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:19.884453 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398a40c-b9e9-4a50-9b71-99a7d0318385 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:19.893066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8e7cc-6689-41c3-86d7-99d2c883166a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:19.895037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c5e8e7cc-6689-41c3-86d7-99d2c883166a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:19.901498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 864/1742] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:25:19.905767 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398a40c-b9e9-4a50-9b71-99a7d0318385 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:19.918067 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8e7cc-6689-41c3-86d7-99d2c883166a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:19.923746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08bf70-d30a-465a-938b-d4016dd552a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:19.923931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398a40c-b9e9-4a50-9b71-99a7d0318385 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:19.925014 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08bf70-d30a-465a-938b-d4016dd552a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:19.925387 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08bf70-d30a-465a-938b-d4016dd552a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:19.925761 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08bf70-d30a-465a-938b-d4016dd552a3 admin admin] PATH_INFO: `/v3/users/3e2eb1d3dd62412d8569c41d141e01cd` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:19.927188 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08bf70-d30a-465a-938b-d4016dd552a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e2eb1d3dd62412d8569c41d141e01cd)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:19.929231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08bf70-d30a-465a-938b-d4016dd552a3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:19.929485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f90b0-8551-44bc-84c2-1609f2132b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:19.930097 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f90b0-8551-44bc-84c2-1609f2132b0c admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:19.930376 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f90b0-8551-44bc-84c2-1609f2132b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:19.930643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f90b0-8551-44bc-84c2-1609f2132b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:19.931406 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f90b0-8551-44bc-84c2-1609f2132b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:19.933643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f90b0-8551-44bc-84c2-1609f2132b0c admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:19.944263 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 865/1743] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:19.951781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f90b0-8551-44bc-84c2-1609f2132b0c admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:19.962945 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f90b0-8551-44bc-84c2-1609f2132b0c admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:19.968106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011995d1-5ec1-4595-8a01-4495f8526206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:19.968860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011995d1-5ec1-4595-8a01-4495f8526206 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:19.969209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011995d1-5ec1-4595-8a01-4495f8526206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:19.969584 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011995d1-5ec1-4595-8a01-4495f8526206 admin admin] PATH_INFO: `/v3/projects/be855ec94d234ccab323c123b8ed0aa7/users/348a742d8d064e60b9112e855a990ea3/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:19.987213 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011995d1-5ec1-4595-8a01-4495f8526206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be855ec94d234ccab323c123b8ed0aa7, user_id=348a742d8d064e60b9112e855a990ea3, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:19.990020 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011995d1-5ec1-4595-8a01-4495f8526206 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:20.178426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 866/1744] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:19 2022] PUT /identity/v3/projects/be855ec94d234ccab323c123b8ed0aa7/users/348a742d8d064e60b9112e855a990ea3/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:25:20.186567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011995d1-5ec1-4595-8a01-4495f8526206 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:20.186567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-011995d1-5ec1-4595-8a01-4495f8526206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:20.287845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3b08bf70-d30a-465a-938b-d4016dd552a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e2eb1d3dd62412d8569c41d141e01cd'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:20.299336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011995d1-5ec1-4595-8a01-4495f8526206 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:20.310500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fe3b9-d1c2-4e01-aa21-48b755fe3bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:20.311491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fe3b9-d1c2-4e01-aa21-48b755fe3bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:20.311940 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fe3b9-d1c2-4e01-aa21-48b755fe3bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:20.312529 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fe3b9-d1c2-4e01-aa21-48b755fe3bae admin admin] PATH_INFO: `/v3/projects/f9807c47306440cba6a7131690dfd662` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:20.315097 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fe3b9-d1c2-4e01-aa21-48b755fe3bae admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9807c47306440cba6a7131690dfd662)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:20.318582 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fe3b9-d1c2-4e01-aa21-48b755fe3bae admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:20.320387 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3b08bf70-d30a-465a-938b-d4016dd552a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e2eb1d3dd62412d8569c41d141e01cd'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:20.335830 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3b08bf70-d30a-465a-938b-d4016dd552a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e2eb1d3dd62412d8569c41d141e01cd'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:20.454911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:20.487711 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3b08bf70-d30a-465a-938b-d4016dd552a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e2eb1d3dd62412d8569c41d141e01cd'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:20.488358 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3b08bf70-d30a-465a-938b-d4016dd552a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e2eb1d3dd62412d8569c41d141e01cd'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:20.501551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 879/1745] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:19 2022] DELETE /identity/v3/users/3e2eb1d3dd62412d8569c41d141e01cd => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:20.502787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08bf70-d30a-465a-938b-d4016dd552a3 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:20.507461 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3b08bf70-d30a-465a-938b-d4016dd552a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:20.554969 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e11fe3b9-d1c2-4e01-aa21-48b755fe3bae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9807c47306440cba6a7131690dfd662', 'request_id': 'req-e11fe3b9-d1c2-4e01-aa21-48b755fe3bae'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:20.576552 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e11fe3b9-d1c2-4e01-aa21-48b755fe3bae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9807c47306440cba6a7131690dfd662', 'request_id': 'req-e11fe3b9-d1c2-4e01-aa21-48b755fe3bae'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:20.689480 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e11fe3b9-d1c2-4e01-aa21-48b755fe3bae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9807c47306440cba6a7131690dfd662', 'request_id': 'req-e11fe3b9-d1c2-4e01-aa21-48b755fe3bae'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:20.713144 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e11fe3b9-d1c2-4e01-aa21-48b755fe3bae admin admin] The token cache is being invalidate because project f9807c47306440cba6a7131690dfd662 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:20.715257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e11fe3b9-d1c2-4e01-aa21-48b755fe3bae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:20.718221 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e11fe3b9-d1c2-4e01-aa21-48b755fe3bae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:20.724400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 867/1746] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:20 2022] DELETE /identity/v3/projects/f9807c47306440cba6a7131690dfd662 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:20.725414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b492dc-4a55-4f03-aa85-690f9a8749d8 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:20.725705 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b492dc-4a55-4f03-aa85-690f9a8749d8 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:20.725991 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b492dc-4a55-4f03-aa85-690f9a8749d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:20.735290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:20.767637 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08bf70-d30a-465a-938b-d4016dd552a3 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:20.794968 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-61b492dc-4a55-4f03-aa85-690f9a8749d8 None None] MFA Rules not processed for user `348a742d8d064e60b9112e855a990ea3`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:20.798812 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-61b492dc-4a55-4f03-aa85-690f9a8749d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:20.951247 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011f716-f1e3-4f1a-9dbc-a7a8aa9888ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:20.955296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011f716-f1e3-4f1a-9dbc-a7a8aa9888ab service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:20.955296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011f716-f1e3-4f1a-9dbc-a7a8aa9888ab service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:20.955296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011f716-f1e3-4f1a-9dbc-a7a8aa9888ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:20.958984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7011f716-f1e3-4f1a-9dbc-a7a8aa9888ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:20.983371 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 868/1747] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:20.984638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b492dc-4a55-4f03-aa85-690f9a8749d8 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:20.988076 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-61b492dc-4a55-4f03-aa85-690f9a8749d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:20.996963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011f716-f1e3-4f1a-9dbc-a7a8aa9888ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:21.004556 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011f716-f1e3-4f1a-9dbc-a7a8aa9888ab service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:21.033131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 880/1748] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:21.038508 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae96054-efab-40ff-855d-27d880a5d0bc None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:21.039071 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae96054-efab-40ff-855d-27d880a5d0bc None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:21.040537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae96054-efab-40ff-855d-27d880a5d0bc None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:21.044013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 881/1749] 10.222.0.78 () {54 vars in 921 bytes} [Mon Sep 19 11:25:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:25:21.046639 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2c47a-3146-4cac-979f-da1c57b67ae3 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:21.048091 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2c47a-3146-4cac-979f-da1c57b67ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:21.049129 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2c47a-3146-4cac-979f-da1c57b67ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:21.104064 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b2d2c47a-3146-4cac-979f-da1c57b67ae3 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:21.107799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b2d2c47a-3146-4cac-979f-da1c57b67ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:21.175741 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b492dc-4a55-4f03-aa85-690f9a8749d8 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:21.193814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cb4c9-8dae-4cc8-b310-14d57d5dab90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:21.197213 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cb4c9-8dae-4cc8-b310-14d57d5dab90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:21.199994 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cb4c9-8dae-4cc8-b310-14d57d5dab90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:21.200915 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cb4c9-8dae-4cc8-b310-14d57d5dab90 admin admin] PATH_INFO: `/v3/projects/7baafab3509c49fa96dcf385ff195aba` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:21.205663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668cb4c9-8dae-4cc8-b310-14d57d5dab90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7baafab3509c49fa96dcf385ff195aba)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:21.222478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668cb4c9-8dae-4cc8-b310-14d57d5dab90 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:21.222796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 882/1750] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:21.223141 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d2c47a-3146-4cac-979f-da1c57b67ae3 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:21.228254 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b2d2c47a-3146-4cac-979f-da1c57b67ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:21.268647 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d2c47a-3146-4cac-979f-da1c57b67ae3 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:21.279588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5e4b5-bb5f-4824-aca0-1e880edb416d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:21.281162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5e4b5-bb5f-4824-aca0-1e880edb416d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:21.281642 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5e4b5-bb5f-4824-aca0-1e880edb416d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:21.282336 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5e4b5-bb5f-4824-aca0-1e880edb416d admin admin] PATH_INFO: `/v3/users/bc89f22ac62749f0a1d85097a743307e` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:21.286453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5e4b5-bb5f-4824-aca0-1e880edb416d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc89f22ac62749f0a1d85097a743307e)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:21.293954 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5e4b5-bb5f-4824-aca0-1e880edb416d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:21.444379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-668cb4c9-8dae-4cc8-b310-14d57d5dab90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7baafab3509c49fa96dcf385ff195aba', 'request_id': 'req-668cb4c9-8dae-4cc8-b310-14d57d5dab90'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:21.461767 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-668cb4c9-8dae-4cc8-b310-14d57d5dab90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7baafab3509c49fa96dcf385ff195aba', 'request_id': 'req-668cb4c9-8dae-4cc8-b310-14d57d5dab90'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:21.592966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-668cb4c9-8dae-4cc8-b310-14d57d5dab90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7baafab3509c49fa96dcf385ff195aba', 'request_id': 'req-668cb4c9-8dae-4cc8-b310-14d57d5dab90'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:21.621673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-78d5e4b5-bb5f-4824-aca0-1e880edb416d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc89f22ac62749f0a1d85097a743307e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:21.628025 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-668cb4c9-8dae-4cc8-b310-14d57d5dab90 admin admin] The token cache is being invalidate because project 7baafab3509c49fa96dcf385ff195aba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:21.628653 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-668cb4c9-8dae-4cc8-b310-14d57d5dab90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:21.629740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-668cb4c9-8dae-4cc8-b310-14d57d5dab90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:21.634253 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 869/1751] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:20 2022] DELETE /identity/v3/projects/7baafab3509c49fa96dcf385ff195aba => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:21.637856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cb4c9-8dae-4cc8-b310-14d57d5dab90 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:21.648175 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-668cb4c9-8dae-4cc8-b310-14d57d5dab90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:21.689873 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-78d5e4b5-bb5f-4824-aca0-1e880edb416d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc89f22ac62749f0a1d85097a743307e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:21.714774 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-78d5e4b5-bb5f-4824-aca0-1e880edb416d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc89f22ac62749f0a1d85097a743307e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:21.813913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-78d5e4b5-bb5f-4824-aca0-1e880edb416d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc89f22ac62749f0a1d85097a743307e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:21.815013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-78d5e4b5-bb5f-4824-aca0-1e880edb416d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc89f22ac62749f0a1d85097a743307e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:21.828563 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 883/1752] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:21 2022] DELETE /identity/v3/users/bc89f22ac62749f0a1d85097a743307e => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:21.829864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f761b-a28d-4525-ab43-fb920ef631a6 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:21.830199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f761b-a28d-4525-ab43-fb920ef631a6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:21.837211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f761b-a28d-4525-ab43-fb920ef631a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:21.895228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b54f761b-a28d-4525-ab43-fb920ef631a6 None None] MFA Rules not processed for user `aba1968529574fe5b4c5d114c874f428`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:21.907845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b54f761b-a28d-4525-ab43-fb920ef631a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:21.932652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cb4c9-8dae-4cc8-b310-14d57d5dab90 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:22.080326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8305c70-43dd-417b-ad8b-b5ee3790803a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:22.081041 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8305c70-43dd-417b-ad8b-b5ee3790803a admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:22.081499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8305c70-43dd-417b-ad8b-b5ee3790803a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:22.082258 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8305c70-43dd-417b-ad8b-b5ee3790803a admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:22.082964 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8305c70-43dd-417b-ad8b-b5ee3790803a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:22.085230 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8305c70-43dd-417b-ad8b-b5ee3790803a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:22.187216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 884/1753] 10.222.0.78 () {58 vars in 1018 bytes} [Mon Sep 19 11:25:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:22.188239 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f761b-a28d-4525-ab43-fb920ef631a6 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:22.190386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b54f761b-a28d-4525-ab43-fb920ef631a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:22.344119 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f761b-a28d-4525-ab43-fb920ef631a6 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:22.354888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309605e-a16d-47a8-8420-b32c52a6af50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:22.354888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309605e-a16d-47a8-8420-b32c52a6af50 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:22.354888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309605e-a16d-47a8-8420-b32c52a6af50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:22.354888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309605e-a16d-47a8-8420-b32c52a6af50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:22.356953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6309605e-a16d-47a8-8420-b32c52a6af50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:22.359636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 870/1754] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:21 2022] POST /identity/v3/projects => generated 373 bytes in 725 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:22.364360 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77913df-740e-4805-a787-aa9d7ce98b00 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:22.366792 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77913df-740e-4805-a787-aa9d7ce98b00 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:22.367650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77913df-740e-4805-a787-aa9d7ce98b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:22.384073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309605e-a16d-47a8-8420-b32c52a6af50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:22.385801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309605e-a16d-47a8-8420-b32c52a6af50 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:22.418316 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 885/1755] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:22.420141 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309605e-a16d-47a8-8420-b32c52a6af50 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:22.423991 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6309605e-a16d-47a8-8420-b32c52a6af50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:22.426616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-e77913df-740e-4805-a787-aa9d7ce98b00 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:22.429165 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e77913df-740e-4805-a787-aa9d7ce98b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:22.473890 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309605e-a16d-47a8-8420-b32c52a6af50 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:22.613860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 871/1756] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:22.615866 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77913df-740e-4805-a787-aa9d7ce98b00 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:22.617579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e77913df-740e-4805-a787-aa9d7ce98b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:22.648609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77913df-740e-4805-a787-aa9d7ce98b00 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:22.656026 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d0944-de21-43eb-a7c5-4f146b96d82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:22.656861 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d0944-de21-43eb-a7c5-4f146b96d82a admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:22.657272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d0944-de21-43eb-a7c5-4f146b96d82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:22.657647 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d0944-de21-43eb-a7c5-4f146b96d82a admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:22.658508 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d0944-de21-43eb-a7c5-4f146b96d82a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:22.661302 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d0944-de21-43eb-a7c5-4f146b96d82a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:22.719561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960c7cd-ad83-4b13-98e3-655d8c91af54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:22.719561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960c7cd-ad83-4b13-98e3-655d8c91af54 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:22.719561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960c7cd-ad83-4b13-98e3-655d8c91af54 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:22.719561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960c7cd-ad83-4b13-98e3-655d8c91af54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:22.720195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960c7cd-ad83-4b13-98e3-655d8c91af54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:22.720195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960c7cd-ad83-4b13-98e3-655d8c91af54 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:22.720195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 886/1757] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:22.720195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263ddf5-78be-44d5-a5db-eb4c57b3f8a8 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:22.720195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263ddf5-78be-44d5-a5db-eb4c57b3f8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:22.720195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263ddf5-78be-44d5-a5db-eb4c57b3f8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:22.738615 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f263ddf5-78be-44d5-a5db-eb4c57b3f8a8 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:22.740187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f263ddf5-78be-44d5-a5db-eb4c57b3f8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:22.751593 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 887/1758] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:22.756496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263ddf5-78be-44d5-a5db-eb4c57b3f8a8 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:22.778539 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263ddf5-78be-44d5-a5db-eb4c57b3f8a8 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:22.783771 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc00c5-01f2-4b04-8d1e-8b5d59f8d7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:22.784488 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc00c5-01f2-4b04-8d1e-8b5d59f8d7d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:22.784771 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc00c5-01f2-4b04-8d1e-8b5d59f8d7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:22.785039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc00c5-01f2-4b04-8d1e-8b5d59f8d7d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:22.785694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc00c5-01f2-4b04-8d1e-8b5d59f8d7d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:22.785992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc00c5-01f2-4b04-8d1e-8b5d59f8d7d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:22.787707 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc00c5-01f2-4b04-8d1e-8b5d59f8d7d5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:22.800408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 888/1759] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:25:22.804779 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc00c5-01f2-4b04-8d1e-8b5d59f8d7d5 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:22.815797 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc00c5-01f2-4b04-8d1e-8b5d59f8d7d5 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:22.820810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc340f0a-b910-450f-a756-de0d9f61b987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:22.821487 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc340f0a-b910-450f-a756-de0d9f61b987 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:22.821836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc340f0a-b910-450f-a756-de0d9f61b987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:22.822215 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc340f0a-b910-450f-a756-de0d9f61b987 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:22.822735 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc340f0a-b910-450f-a756-de0d9f61b987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:22.825433 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc340f0a-b910-450f-a756-de0d9f61b987 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:23.355150 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 872/1760] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:22 2022] POST /identity/v3/users => generated 293 bytes in 740 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:23.356901 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d0944-de21-43eb-a7c5-4f146b96d82a admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:23.366844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 889/1761] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:22 2022] POST /identity/v3/projects => generated 381 bytes in 562 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:25:23.371440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc340f0a-b910-450f-a756-de0d9f61b987 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:23.392814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d0944-de21-43eb-a7c5-4f146b96d82a admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:23.546023 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc340f0a-b910-450f-a756-de0d9f61b987 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:23.565538 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c9406-912c-440c-8efd-61da2bbe3137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:23.566209 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c9406-912c-440c-8efd-61da2bbe3137 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:23.566527 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c9406-912c-440c-8efd-61da2bbe3137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:23.566880 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c9406-912c-440c-8efd-61da2bbe3137 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:23.572325 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c9406-912c-440c-8efd-61da2bbe3137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:23.572325 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c9406-912c-440c-8efd-61da2bbe3137 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:23.632255 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd97f03-ee6e-4b70-b06d-43dff168cb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:23.637953 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd97f03-ee6e-4b70-b06d-43dff168cb5f admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:23.640603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd97f03-ee6e-4b70-b06d-43dff168cb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:23.641348 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd97f03-ee6e-4b70-b06d-43dff168cb5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:23.647603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd97f03-ee6e-4b70-b06d-43dff168cb5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:23.649938 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd97f03-ee6e-4b70-b06d-43dff168cb5f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:23.679490 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 873/1762] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:23.685183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd97f03-ee6e-4b70-b06d-43dff168cb5f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:23.688218 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7dd97f03-ee6e-4b70-b06d-43dff168cb5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:23.721600 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd97f03-ee6e-4b70-b06d-43dff168cb5f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:23.732407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4caea-5599-43e0-8223-9f15096685fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:23.736655 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4caea-5599-43e0-8223-9f15096685fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:23.737302 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4caea-5599-43e0-8223-9f15096685fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:23.737588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4caea-5599-43e0-8223-9f15096685fc admin admin] PATH_INFO: `/v3/projects/63d66661196446c9aa98d08e833b286b` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:23.743258 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4caea-5599-43e0-8223-9f15096685fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=63d66661196446c9aa98d08e833b286b)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:23.745745 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4caea-5599-43e0-8223-9f15096685fc admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:23.805822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 890/1763] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:23 2022] POST /identity/v3/users => generated 291 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:23.806665 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c9406-912c-440c-8efd-61da2bbe3137 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:23.837236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c9406-912c-440c-8efd-61da2bbe3137 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:23.853704 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c9ffe-f138-4319-bea5-0c70806ad33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:23.857473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c9ffe-f138-4319-bea5-0c70806ad33d admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:23.857473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c9ffe-f138-4319-bea5-0c70806ad33d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:23.857473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c9ffe-f138-4319-bea5-0c70806ad33d admin admin] PATH_INFO: `/v3/projects/0a0f3fb845c04fe98e7c76352902436a/users/b91aaad8479b4c189ee0d43fc2db6dee/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:23.917290 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c9ffe-f138-4319-bea5-0c70806ad33d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a0f3fb845c04fe98e7c76352902436a, user_id=b91aaad8479b4c189ee0d43fc2db6dee, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:23.926905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c9ffe-f138-4319-bea5-0c70806ad33d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:24.060385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 891/1764] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:23 2022] PUT /identity/v3/projects/0a0f3fb845c04fe98e7c76352902436a/users/b91aaad8479b4c189ee0d43fc2db6dee/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:24.061708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c9ffe-f138-4319-bea5-0c70806ad33d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:24.088546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-bab4caea-5599-43e0-8223-9f15096685fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63d66661196446c9aa98d08e833b286b', 'request_id': 'req-bab4caea-5599-43e0-8223-9f15096685fc'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:24.116586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-bab4caea-5599-43e0-8223-9f15096685fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63d66661196446c9aa98d08e833b286b', 'request_id': 'req-bab4caea-5599-43e0-8223-9f15096685fc'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:24.262095 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c9ffe-f138-4319-bea5-0c70806ad33d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:24.276556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-bab4caea-5599-43e0-8223-9f15096685fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63d66661196446c9aa98d08e833b286b', 'request_id': 'req-bab4caea-5599-43e0-8223-9f15096685fc'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:24.295887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-bab4caea-5599-43e0-8223-9f15096685fc admin admin] The token cache is being invalidate because project 63d66661196446c9aa98d08e833b286b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:24.300522 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-bab4caea-5599-43e0-8223-9f15096685fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:24.305308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-bab4caea-5599-43e0-8223-9f15096685fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:24.312357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 874/1765] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:23 2022] DELETE /identity/v3/projects/63d66661196446c9aa98d08e833b286b => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:24.313802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4caea-5599-43e0-8223-9f15096685fc admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:24.320519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-bab4caea-5599-43e0-8223-9f15096685fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:24.464015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8ffa5-9ead-42e0-a6c9-78cfa1042591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:24.468142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8ffa5-9ead-42e0-a6c9-78cfa1042591 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:24.469279 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8ffa5-9ead-42e0-a6c9-78cfa1042591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:24.470561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8ffa5-9ead-42e0-a6c9-78cfa1042591 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:24.473483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8ffa5-9ead-42e0-a6c9-78cfa1042591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:24.480481 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8ffa5-9ead-42e0-a6c9-78cfa1042591 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:24.501877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4caea-5599-43e0-8223-9f15096685fc admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:24.513211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d6fdc-60bf-4e65-bdd6-a08ec2400821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:24.514084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 892/1766] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:24.514915 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d6fdc-60bf-4e65-bdd6-a08ec2400821 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:24.515338 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d6fdc-60bf-4e65-bdd6-a08ec2400821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:24.517433 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249f5ec-9471-43fb-ba8d-ff8346f5c1b6 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:24.517711 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249f5ec-9471-43fb-ba8d-ff8346f5c1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:24.517984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249f5ec-9471-43fb-ba8d-ff8346f5c1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:24.520664 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d6fdc-60bf-4e65-bdd6-a08ec2400821 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:24.521530 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d6fdc-60bf-4e65-bdd6-a08ec2400821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:24.523256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d6fdc-60bf-4e65-bdd6-a08ec2400821 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:24.550954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 875/1767] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:24.553340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d6fdc-60bf-4e65-bdd6-a08ec2400821 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:24.558702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-8d3d6fdc-60bf-4e65-bdd6-a08ec2400821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:24.567431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-3249f5ec-9471-43fb-ba8d-ff8346f5c1b6 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:24.572465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3249f5ec-9471-43fb-ba8d-ff8346f5c1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:24.588413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d6fdc-60bf-4e65-bdd6-a08ec2400821 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:24.601221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 893/1768] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:24.602682 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249f5ec-9471-43fb-ba8d-ff8346f5c1b6 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:24.612805 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb89689-7317-4934-b99c-dce310e4d841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:24.613064 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb89689-7317-4934-b99c-dce310e4d841 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:24.613409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb89689-7317-4934-b99c-dce310e4d841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:24.613681 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb89689-7317-4934-b99c-dce310e4d841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:24.615597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb89689-7317-4934-b99c-dce310e4d841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:24.617271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb89689-7317-4934-b99c-dce310e4d841 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:24.625184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249f5ec-9471-43fb-ba8d-ff8346f5c1b6 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:24.638212 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf82ca2-7019-4f05-b171-1760b7336826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:24.639116 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf82ca2-7019-4f05-b171-1760b7336826 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:24.639400 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf82ca2-7019-4f05-b171-1760b7336826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:24.639917 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf82ca2-7019-4f05-b171-1760b7336826 admin admin] PATH_INFO: `/v3/projects/0a0f3fb845c04fe98e7c76352902436a/users/b91aaad8479b4c189ee0d43fc2db6dee/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:24.641658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf82ca2-7019-4f05-b171-1760b7336826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a0f3fb845c04fe98e7c76352902436a, user_id=b91aaad8479b4c189ee0d43fc2db6dee, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:24.644505 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf82ca2-7019-4f05-b171-1760b7336826 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:24.656078 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 876/1769] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:24.656078 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4986b8-2531-4f14-9f6a-7cfe0baacea0 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:24.656078 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4986b8-2531-4f14-9f6a-7cfe0baacea0 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:24.656078 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4986b8-2531-4f14-9f6a-7cfe0baacea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:24.687574 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 894/1770] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:24 2022] PUT /identity/v3/projects/0a0f3fb845c04fe98e7c76352902436a/users/b91aaad8479b4c189ee0d43fc2db6dee/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:24.687574 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf82ca2-7019-4f05-b171-1760b7336826 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:24.701198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-5a4986b8-2531-4f14-9f6a-7cfe0baacea0 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:24.704126 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5a4986b8-2531-4f14-9f6a-7cfe0baacea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:24.882519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf82ca2-7019-4f05-b171-1760b7336826 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:24.886417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 877/1771] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:24.886417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6591c-108e-4e3e-99a7-cc9d902f6263 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:24.886417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6591c-108e-4e3e-99a7-cc9d902f6263 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:24.886417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6591c-108e-4e3e-99a7-cc9d902f6263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:24.893826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b9896-560e-4be3-899a-3c00876506e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:24.894650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b9896-560e-4be3-899a-3c00876506e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:24.895029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b9896-560e-4be3-899a-3c00876506e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:24.895453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b9896-560e-4be3-899a-3c00876506e2 admin admin] PATH_INFO: `/v3/projects/3bead1f8c4e64e128d92f2c734d456c5/users/6936cac7122343938b41c19c961850a5/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:24.917918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b9896-560e-4be3-899a-3c00876506e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bead1f8c4e64e128d92f2c734d456c5, user_id=6936cac7122343938b41c19c961850a5, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:24.921498 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b9896-560e-4be3-899a-3c00876506e2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:24.925339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-38c6591c-108e-4e3e-99a7-cc9d902f6263 None None] MFA Rules not processed for user `b91aaad8479b4c189ee0d43fc2db6dee`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:24.927747 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-38c6591c-108e-4e3e-99a7-cc9d902f6263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:25.022685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 895/1772] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:24 2022] PUT /identity/v3/projects/3bead1f8c4e64e128d92f2c734d456c5/users/6936cac7122343938b41c19c961850a5/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:25.022685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b9896-560e-4be3-899a-3c00876506e2 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:25.088038 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 878/1773] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:25.089551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6591c-108e-4e3e-99a7-cc9d902f6263 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:25.093107 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-38c6591c-108e-4e3e-99a7-cc9d902f6263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:25.120624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6591c-108e-4e3e-99a7-cc9d902f6263 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:25.169895 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b9896-560e-4be3-899a-3c00876506e2 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:25.174628 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194408d0-bb6d-45cb-a42d-26022f33674e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:25.175292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194408d0-bb6d-45cb-a42d-26022f33674e admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:25.175594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194408d0-bb6d-45cb-a42d-26022f33674e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:25.176017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194408d0-bb6d-45cb-a42d-26022f33674e admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:25.176686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194408d0-bb6d-45cb-a42d-26022f33674e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:25.176992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194408d0-bb6d-45cb-a42d-26022f33674e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:25.178686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194408d0-bb6d-45cb-a42d-26022f33674e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:25.200401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 896/1774] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:25:25.202610 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3406f8f-fb94-4910-82aa-50d0fc93b5a7 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:25.203154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3406f8f-fb94-4910-82aa-50d0fc93b5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:25.206544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3406f8f-fb94-4910-82aa-50d0fc93b5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:25.249450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d3406f8f-fb94-4910-82aa-50d0fc93b5a7 None None] MFA Rules not processed for user `6936cac7122343938b41c19c961850a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:25.254170 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d3406f8f-fb94-4910-82aa-50d0fc93b5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:25.264848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee15541e-eaae-48db-a27a-893a205ed515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:25.265593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15541e-eaae-48db-a27a-893a205ed515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:25.265868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15541e-eaae-48db-a27a-893a205ed515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:25.266137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15541e-eaae-48db-a27a-893a205ed515 admin admin] PATH_INFO: `/v3/projects/59bbe0b3d779464db5b6e996d4a4febe` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:25.267499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee15541e-eaae-48db-a27a-893a205ed515 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59bbe0b3d779464db5b6e996d4a4febe)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:25.270665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee15541e-eaae-48db-a27a-893a205ed515 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:25.449878 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 897/1775] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:25.450928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3406f8f-fb94-4910-82aa-50d0fc93b5a7 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:25.459332 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d3406f8f-fb94-4910-82aa-50d0fc93b5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:25.531260 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3406f8f-fb94-4910-82aa-50d0fc93b5a7 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:25.536110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef0a1a-b52a-4b8d-9cf7-b462c04a88bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:25.537041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef0a1a-b52a-4b8d-9cf7-b462c04a88bf service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:25.537535 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef0a1a-b52a-4b8d-9cf7-b462c04a88bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:25.537997 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef0a1a-b52a-4b8d-9cf7-b462c04a88bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:25.540220 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ffef0a1a-b52a-4b8d-9cf7-b462c04a88bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:25.620452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef0a1a-b52a-4b8d-9cf7-b462c04a88bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:25.622027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef0a1a-b52a-4b8d-9cf7-b462c04a88bf service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:25.648131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 898/1776] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:25.648131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef0a1a-b52a-4b8d-9cf7-b462c04a88bf service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:25.671614 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef0a1a-b52a-4b8d-9cf7-b462c04a88bf service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:25.689472 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99901999-004a-4e8c-b7f6-e3b38edf962a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:25.690327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99901999-004a-4e8c-b7f6-e3b38edf962a admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:25.690680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99901999-004a-4e8c-b7f6-e3b38edf962a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:25.691097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99901999-004a-4e8c-b7f6-e3b38edf962a admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:25.691869 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99901999-004a-4e8c-b7f6-e3b38edf962a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:25.713089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99901999-004a-4e8c-b7f6-e3b38edf962a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:25.767585 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:25.798250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ee15541e-eaae-48db-a27a-893a205ed515 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59bbe0b3d779464db5b6e996d4a4febe', 'request_id': 'req-ee15541e-eaae-48db-a27a-893a205ed515'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:25.811483 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ee15541e-eaae-48db-a27a-893a205ed515 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59bbe0b3d779464db5b6e996d4a4febe', 'request_id': 'req-ee15541e-eaae-48db-a27a-893a205ed515'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:25.846447 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 899/1777] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:25 2022] POST /identity/v3/projects => generated 365 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:25.848468 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99901999-004a-4e8c-b7f6-e3b38edf962a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:25.926321 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ee15541e-eaae-48db-a27a-893a205ed515 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59bbe0b3d779464db5b6e996d4a4febe', 'request_id': 'req-ee15541e-eaae-48db-a27a-893a205ed515'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:25.940280 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ee15541e-eaae-48db-a27a-893a205ed515 admin admin] The token cache is being invalidate because project 59bbe0b3d779464db5b6e996d4a4febe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:25.940574 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ee15541e-eaae-48db-a27a-893a205ed515 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:25.941237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ee15541e-eaae-48db-a27a-893a205ed515 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:25.942780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 879/1778] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:25 2022] DELETE /identity/v3/projects/59bbe0b3d779464db5b6e996d4a4febe => generated 0 bytes in 854 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:25.943604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee15541e-eaae-48db-a27a-893a205ed515 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:25.944923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ee15541e-eaae-48db-a27a-893a205ed515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:25.982691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99901999-004a-4e8c-b7f6-e3b38edf962a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:25.991205 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071d81c-55dd-47dc-83ab-d3d980c2c572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:25.991205 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071d81c-55dd-47dc-83ab-d3d980c2c572 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:25.991205 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071d81c-55dd-47dc-83ab-d3d980c2c572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:25.991205 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071d81c-55dd-47dc-83ab-d3d980c2c572 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:25.991676 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071d81c-55dd-47dc-83ab-d3d980c2c572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:25.991899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071d81c-55dd-47dc-83ab-d3d980c2c572 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:26.014474 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee15541e-eaae-48db-a27a-893a205ed515 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:26.021035 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ab4b1-f9e1-4430-9b20-188b72eab5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:26.021870 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ab4b1-f9e1-4430-9b20-188b72eab5f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:26.022253 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ab4b1-f9e1-4430-9b20-188b72eab5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:26.022751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ab4b1-f9e1-4430-9b20-188b72eab5f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:26.023752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ab4b1-f9e1-4430-9b20-188b72eab5f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:26.026752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ab4b1-f9e1-4430-9b20-188b72eab5f8 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:26.137411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 880/1779] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:25 2022] POST /identity/v3/users => generated 283 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:26.138293 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 900/1780] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:25 2022] POST /identity/v3/projects => generated 381 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:26.144880 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071d81c-55dd-47dc-83ab-d3d980c2c572 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:26.147884 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ab4b1-f9e1-4430-9b20-188b72eab5f8 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:26.151802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-cc6ab4b1-f9e1-4430-9b20-188b72eab5f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:26.172132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ab4b1-f9e1-4430-9b20-188b72eab5f8 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:26.231601 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071d81c-55dd-47dc-83ab-d3d980c2c572 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:26.238214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b051a-bfb2-48eb-bc7b-da840d2365e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:26.238969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b051a-bfb2-48eb-bc7b-da840d2365e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:26.239385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b051a-bfb2-48eb-bc7b-da840d2365e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:26.240054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b051a-bfb2-48eb-bc7b-da840d2365e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:26.240988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b051a-bfb2-48eb-bc7b-da840d2365e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:26.243013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b051a-bfb2-48eb-bc7b-da840d2365e9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:26.260081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816180b-44ed-4929-8237-44c5026256e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:26.260341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816180b-44ed-4929-8237-44c5026256e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:26.260753 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816180b-44ed-4929-8237-44c5026256e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:26.260871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816180b-44ed-4929-8237-44c5026256e1 admin admin] PATH_INFO: `/v3/users/d5438ba7ea0e463dac23348fcf55ca6c` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:26.261360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 901/1781] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:26.262633 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b051a-bfb2-48eb-bc7b-da840d2365e9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:26.262932 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816180b-44ed-4929-8237-44c5026256e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5438ba7ea0e463dac23348fcf55ca6c)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:26.264784 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-261b051a-bfb2-48eb-bc7b-da840d2365e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:26.267056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816180b-44ed-4929-8237-44c5026256e1 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:26.283880 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b051a-bfb2-48eb-bc7b-da840d2365e9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:26.297145 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aac23-0f88-4a7c-a1d2-9e0d0e47dd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:26.297851 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aac23-0f88-4a7c-a1d2-9e0d0e47dd3b admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:26.298136 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aac23-0f88-4a7c-a1d2-9e0d0e47dd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:26.298485 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aac23-0f88-4a7c-a1d2-9e0d0e47dd3b admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:26.299122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46aac23-0f88-4a7c-a1d2-9e0d0e47dd3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:26.301168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46aac23-0f88-4a7c-a1d2-9e0d0e47dd3b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:26.954868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 902/1782] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:26 2022] POST /identity/v3/users => generated 297 bytes in 693 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:26.957420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aac23-0f88-4a7c-a1d2-9e0d0e47dd3b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:26.982728 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aac23-0f88-4a7c-a1d2-9e0d0e47dd3b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:26.989730 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a65d4f-a934-4752-ad66-abe2e25f7676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:26.990513 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a65d4f-a934-4752-ad66-abe2e25f7676 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:26.990990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a65d4f-a934-4752-ad66-abe2e25f7676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:26.991300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a65d4f-a934-4752-ad66-abe2e25f7676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:26.992120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a65d4f-a934-4752-ad66-abe2e25f7676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:26.994098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a65d4f-a934-4752-ad66-abe2e25f7676 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:27.025122 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1816180b-44ed-4929-8237-44c5026256e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5438ba7ea0e463dac23348fcf55ca6c'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:27.032273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:27.036263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 903/1783] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:27.036263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a65d4f-a934-4752-ad66-abe2e25f7676 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:27.045977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1816180b-44ed-4929-8237-44c5026256e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5438ba7ea0e463dac23348fcf55ca6c'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:27.054505 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1816180b-44ed-4929-8237-44c5026256e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5438ba7ea0e463dac23348fcf55ca6c'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:27.063681 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a65d4f-a934-4752-ad66-abe2e25f7676 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:27.070233 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a10565-f3d5-4dbd-90bc-bda7130a7cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:27.072195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a10565-f3d5-4dbd-90bc-bda7130a7cff admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:27.072195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a10565-f3d5-4dbd-90bc-bda7130a7cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:27.072195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a10565-f3d5-4dbd-90bc-bda7130a7cff admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:27.073610 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a10565-f3d5-4dbd-90bc-bda7130a7cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:27.075199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a10565-f3d5-4dbd-90bc-bda7130a7cff admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:27.089340 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 904/1784] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:27.090202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a10565-f3d5-4dbd-90bc-bda7130a7cff admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:27.121097 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1816180b-44ed-4929-8237-44c5026256e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5438ba7ea0e463dac23348fcf55ca6c'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:27.121863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1816180b-44ed-4929-8237-44c5026256e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5438ba7ea0e463dac23348fcf55ca6c'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:27.131479 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 881/1785] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:26 2022] DELETE /identity/v3/users/d5438ba7ea0e463dac23348fcf55ca6c => generated 0 bytes in 993 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:27.145292 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816180b-44ed-4929-8237-44c5026256e1 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:27.145292 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1816180b-44ed-4929-8237-44c5026256e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:27.155929 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a10565-f3d5-4dbd-90bc-bda7130a7cff admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:27.311763 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe3b2e-a722-4c35-ac9e-95b2713a5426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:27.312528 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe3b2e-a722-4c35-ac9e-95b2713a5426 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:27.312874 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe3b2e-a722-4c35-ac9e-95b2713a5426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:27.313152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe3b2e-a722-4c35-ac9e-95b2713a5426 admin admin] PATH_INFO: `/v3/projects/fced8c972a7d4ee8aa530835a32bc80a/users/45a6def2b0c442f48af637c0715f3a69/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:27.341864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe3b2e-a722-4c35-ac9e-95b2713a5426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fced8c972a7d4ee8aa530835a32bc80a, user_id=45a6def2b0c442f48af637c0715f3a69, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:27.344857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe3b2e-a722-4c35-ac9e-95b2713a5426 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:27.351022 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816180b-44ed-4929-8237-44c5026256e1 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:27.367873 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f72e5-9b1a-45f7-9f40-b4e8a203a490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:27.368583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f72e5-9b1a-45f7-9f40-b4e8a203a490 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:27.368867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f72e5-9b1a-45f7-9f40-b4e8a203a490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:27.369161 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f72e5-9b1a-45f7-9f40-b4e8a203a490 admin admin] PATH_INFO: `/v3/projects/cffce302cfc34318b20ff6932085637c/users/b3e5dc6cd39c4af5bc6da31041fc8312/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:27.400266 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f72e5-9b1a-45f7-9f40-b4e8a203a490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cffce302cfc34318b20ff6932085637c, user_id=b3e5dc6cd39c4af5bc6da31041fc8312, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:27.403096 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f72e5-9b1a-45f7-9f40-b4e8a203a490 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:27.453221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 905/1786] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:27 2022] PUT /identity/v3/projects/fced8c972a7d4ee8aa530835a32bc80a/users/45a6def2b0c442f48af637c0715f3a69/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:27.463842 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b37da8-e197-46dc-a6b0-b09b8d17ba20 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:27.464130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b37da8-e197-46dc-a6b0-b09b8d17ba20 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:27.464410 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b37da8-e197-46dc-a6b0-b09b8d17ba20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:27.495841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-50b37da8-e197-46dc-a6b0-b09b8d17ba20 None None] MFA Rules not processed for user `45a6def2b0c442f48af637c0715f3a69`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:27.497119 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-50b37da8-e197-46dc-a6b0-b09b8d17ba20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:27.504874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 882/1787] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:27 2022] PUT /identity/v3/projects/cffce302cfc34318b20ff6932085637c/users/b3e5dc6cd39c4af5bc6da31041fc8312/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:27.508702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f72e5-9b1a-45f7-9f40-b4e8a203a490 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:27.620822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 906/1788] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:27.628567 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b37da8-e197-46dc-a6b0-b09b8d17ba20 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:27.628567 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-50b37da8-e197-46dc-a6b0-b09b8d17ba20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:27.648810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b37da8-e197-46dc-a6b0-b09b8d17ba20 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:27.650131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f72e5-9b1a-45f7-9f40-b4e8a203a490 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:27.657163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a2c5f-0ca8-463c-a31c-29976d77f068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:27.657446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a2c5f-0ca8-463c-a31c-29976d77f068 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:27.657935 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a2c5f-0ca8-463c-a31c-29976d77f068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:27.658219 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a2c5f-0ca8-463c-a31c-29976d77f068 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:27.658498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b2b52-5eda-4b77-9403-3e80f6346298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:27.659241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a2c5f-0ca8-463c-a31c-29976d77f068 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:27.659450 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b2b52-5eda-4b77-9403-3e80f6346298 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:27.661562 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b2b52-5eda-4b77-9403-3e80f6346298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:27.661791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a2c5f-0ca8-463c-a31c-29976d77f068 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:27.662158 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b2b52-5eda-4b77-9403-3e80f6346298 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:27.662947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b2b52-5eda-4b77-9403-3e80f6346298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:27.665104 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b2b52-5eda-4b77-9403-3e80f6346298 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:27.685579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 883/1789] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:25:27.687643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b2b52-5eda-4b77-9403-3e80f6346298 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:27.701367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b2b52-5eda-4b77-9403-3e80f6346298 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:27.706562 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debf228-dc2d-4651-9382-4488e5d79d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:27.706897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debf228-dc2d-4651-9382-4488e5d79d3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:27.707208 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debf228-dc2d-4651-9382-4488e5d79d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:27.707954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debf228-dc2d-4651-9382-4488e5d79d3b admin admin] PATH_INFO: `/v3/projects/cffce302cfc34318b20ff6932085637c/users/b3e5dc6cd39c4af5bc6da31041fc8312/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:27.709739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debf228-dc2d-4651-9382-4488e5d79d3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cffce302cfc34318b20ff6932085637c, user_id=b3e5dc6cd39c4af5bc6da31041fc8312, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:27.712598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debf228-dc2d-4651-9382-4488e5d79d3b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:27.735207 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 884/1790] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:27 2022] PUT /identity/v3/projects/cffce302cfc34318b20ff6932085637c/users/b3e5dc6cd39c4af5bc6da31041fc8312/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:27.738196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79bd72-1be7-4ea3-a701-a04f0d36c206 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:27.738591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79bd72-1be7-4ea3-a701-a04f0d36c206 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:27.738976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79bd72-1be7-4ea3-a701-a04f0d36c206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:27.790644 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-ec79bd72-1be7-4ea3-a701-a04f0d36c206 None None] MFA Rules not processed for user `b3e5dc6cd39c4af5bc6da31041fc8312`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:27.792922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ec79bd72-1be7-4ea3-a701-a04f0d36c206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:27.798761 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 907/1791] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:27 2022] POST /identity/v3/projects => generated 365 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:25:27.802329 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a2c5f-0ca8-463c-a31c-29976d77f068 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:27.937343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a2c5f-0ca8-463c-a31c-29976d77f068 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:27.956519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 885/1792] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:27.956974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e73bb4-e551-4269-ae1c-deff70bee96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:27.957681 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e73bb4-e551-4269-ae1c-deff70bee96f admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:27.957963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e73bb4-e551-4269-ae1c-deff70bee96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:27.958233 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e73bb4-e551-4269-ae1c-deff70bee96f admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:27.958864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e73bb4-e551-4269-ae1c-deff70bee96f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:27.960893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e73bb4-e551-4269-ae1c-deff70bee96f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:27.970532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79bd72-1be7-4ea3-a701-a04f0d36c206 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:27.972226 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ec79bd72-1be7-4ea3-a701-a04f0d36c206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:28.053107 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79bd72-1be7-4ea3-a701-a04f0d36c206 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:28.057343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3848aa8-c23e-45d7-86bc-dbc4b1045e0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:28.057931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3848aa8-c23e-45d7-86bc-dbc4b1045e0a service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:28.058241 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3848aa8-c23e-45d7-86bc-dbc4b1045e0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:28.058520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3848aa8-c23e-45d7-86bc-dbc4b1045e0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:28.075419 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3848aa8-c23e-45d7-86bc-dbc4b1045e0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:28.077314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3848aa8-c23e-45d7-86bc-dbc4b1045e0a service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:28.089362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 886/1793] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:25:28.090302 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3848aa8-c23e-45d7-86bc-dbc4b1045e0a service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:28.094157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f3848aa8-c23e-45d7-86bc-dbc4b1045e0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:28.103431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 908/1794] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:27 2022] POST /identity/v3/users => generated 289 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:28.106800 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e73bb4-e551-4269-ae1c-deff70bee96f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:28.116735 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3848aa8-c23e-45d7-86bc-dbc4b1045e0a service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:28.122265 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e73bb4-e551-4269-ae1c-deff70bee96f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:28.129630 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15f7e2-bf6a-4cde-85db-16e2ceb36099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:28.131052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15f7e2-bf6a-4cde-85db-16e2ceb36099 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:28.131388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15f7e2-bf6a-4cde-85db-16e2ceb36099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:28.131666 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15f7e2-bf6a-4cde-85db-16e2ceb36099 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:28.132388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15f7e2-bf6a-4cde-85db-16e2ceb36099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:28.134082 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15f7e2-bf6a-4cde-85db-16e2ceb36099 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:28.137245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a643a8-3dea-45af-88a4-0e5d9e4f56af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:28.138088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a643a8-3dea-45af-88a4-0e5d9e4f56af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:28.138473 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a643a8-3dea-45af-88a4-0e5d9e4f56af admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:28.139266 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a643a8-3dea-45af-88a4-0e5d9e4f56af admin admin] PATH_INFO: `/v3/projects/141b57f0e32442f7b234c1f41f612c5b` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:28.144095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a643a8-3dea-45af-88a4-0e5d9e4f56af admin admin] RBAC: Authorizing `identity:delete_project(project_id=141b57f0e32442f7b234c1f41f612c5b)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:28.148816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a643a8-3dea-45af-88a4-0e5d9e4f56af admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:28.148959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 909/1795] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:28.155992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15f7e2-bf6a-4cde-85db-16e2ceb36099 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:28.169451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15f7e2-bf6a-4cde-85db-16e2ceb36099 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:28.177965 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad18a0-6181-45bc-854b-57e30a1d52eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:28.183421 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad18a0-6181-45bc-854b-57e30a1d52eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:28.183421 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad18a0-6181-45bc-854b-57e30a1d52eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:28.183421 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad18a0-6181-45bc-854b-57e30a1d52eb admin admin] PATH_INFO: `/v3/projects/1a10385d8c3743c3bc48cc3b39b53e0d/users/51e5d7e02f494ef2bb0bb9fa165f895e/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:28.202201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad18a0-6181-45bc-854b-57e30a1d52eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a10385d8c3743c3bc48cc3b39b53e0d, user_id=51e5d7e02f494ef2bb0bb9fa165f895e, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:28.205330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad18a0-6181-45bc-854b-57e30a1d52eb admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:28.309249 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 910/1796] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:28 2022] PUT /identity/v3/projects/1a10385d8c3743c3bc48cc3b39b53e0d/users/51e5d7e02f494ef2bb0bb9fa165f895e/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:28.312784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-22a643a8-3dea-45af-88a4-0e5d9e4f56af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '141b57f0e32442f7b234c1f41f612c5b', 'request_id': 'req-22a643a8-3dea-45af-88a4-0e5d9e4f56af'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:28.313277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad18a0-6181-45bc-854b-57e30a1d52eb admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:28.322871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-22a643a8-3dea-45af-88a4-0e5d9e4f56af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '141b57f0e32442f7b234c1f41f612c5b', 'request_id': 'req-22a643a8-3dea-45af-88a4-0e5d9e4f56af'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:28.386378 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad18a0-6181-45bc-854b-57e30a1d52eb admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:28.467828 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe29e18-3634-4cf1-884f-8f3791cc17bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:28.468479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe29e18-3634-4cf1-884f-8f3791cc17bd admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:28.468774 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe29e18-3634-4cf1-884f-8f3791cc17bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:28.469025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe29e18-3634-4cf1-884f-8f3791cc17bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:28.477416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe29e18-3634-4cf1-884f-8f3791cc17bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:28.477416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe29e18-3634-4cf1-884f-8f3791cc17bd admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:28.482213 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-22a643a8-3dea-45af-88a4-0e5d9e4f56af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '141b57f0e32442f7b234c1f41f612c5b', 'request_id': 'req-22a643a8-3dea-45af-88a4-0e5d9e4f56af'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:28.498862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-22a643a8-3dea-45af-88a4-0e5d9e4f56af admin admin] The token cache is being invalidate because project 141b57f0e32442f7b234c1f41f612c5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:28.499439 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-22a643a8-3dea-45af-88a4-0e5d9e4f56af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:28.500921 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-22a643a8-3dea-45af-88a4-0e5d9e4f56af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:28.506243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 887/1797] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:28 2022] DELETE /identity/v3/projects/141b57f0e32442f7b234c1f41f612c5b => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:28.506413 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 911/1798] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:25:28.508365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a643a8-3dea-45af-88a4-0e5d9e4f56af admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:28.510017 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-22a643a8-3dea-45af-88a4-0e5d9e4f56af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:28.510187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098caf67-6466-411e-ac7c-34bbb549de1d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:28.510560 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098caf67-6466-411e-ac7c-34bbb549de1d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:28.510910 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098caf67-6466-411e-ac7c-34bbb549de1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:28.528344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a643a8-3dea-45af-88a4-0e5d9e4f56af admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:28.534981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aacf7ba-60c1-4e49-9532-5be6d8c23c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:28.536129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacf7ba-60c1-4e49-9532-5be6d8c23c7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:28.536646 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacf7ba-60c1-4e49-9532-5be6d8c23c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:28.537096 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacf7ba-60c1-4e49-9532-5be6d8c23c7f admin admin] PATH_INFO: `/v3/projects/1a10385d8c3743c3bc48cc3b39b53e0d/users/51e5d7e02f494ef2bb0bb9fa165f895e/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:28.539406 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aacf7ba-60c1-4e49-9532-5be6d8c23c7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a10385d8c3743c3bc48cc3b39b53e0d, user_id=51e5d7e02f494ef2bb0bb9fa165f895e, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:28.544028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aacf7ba-60c1-4e49-9532-5be6d8c23c7f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:28.548554 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-098caf67-6466-411e-ac7c-34bbb549de1d None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:28.551680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-098caf67-6466-411e-ac7c-34bbb549de1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:28.577184 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 888/1799] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:28 2022] PUT /identity/v3/projects/1a10385d8c3743c3bc48cc3b39b53e0d/users/51e5d7e02f494ef2bb0bb9fa165f895e/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:28.589217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b88db-a284-4fb7-b4e3-b11c189f4aaa None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:28.589509 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b88db-a284-4fb7-b4e3-b11c189f4aaa None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:28.589790 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b88db-a284-4fb7-b4e3-b11c189f4aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:28.634138 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 912/1800] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:28.635789 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-543b88db-a284-4fb7-b4e3-b11c189f4aaa None None] MFA Rules not processed for user `51e5d7e02f494ef2bb0bb9fa165f895e`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:28.637285 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-543b88db-a284-4fb7-b4e3-b11c189f4aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:28.642674 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da870f90-7bc1-4920-8db1-426711374b57 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:28.642971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da870f90-7bc1-4920-8db1-426711374b57 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:28.643421 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da870f90-7bc1-4920-8db1-426711374b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:28.664724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-da870f90-7bc1-4920-8db1-426711374b57 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:28.667052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-da870f90-7bc1-4920-8db1-426711374b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:28.675315 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 913/1801] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:28.679646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da870f90-7bc1-4920-8db1-426711374b57 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:28.690755 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da870f90-7bc1-4920-8db1-426711374b57 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:28.696048 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8181884-b133-43de-9877-d8988845b82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:28.697667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8181884-b133-43de-9877-d8988845b82d admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:28.697993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8181884-b133-43de-9877-d8988845b82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:28.698268 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8181884-b133-43de-9877-d8988845b82d admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:28.698952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8181884-b133-43de-9877-d8988845b82d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:28.699298 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8181884-b133-43de-9877-d8988845b82d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:28.700929 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8181884-b133-43de-9877-d8988845b82d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:28.713053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 914/1802] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:25:28.718526 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8181884-b133-43de-9877-d8988845b82d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:28.718714 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 889/1803] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:28.728817 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8181884-b133-43de-9877-d8988845b82d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:28.734075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f17f0-69fd-49ec-93b0-66333b12198c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:28.734670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f17f0-69fd-49ec-93b0-66333b12198c admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:28.734948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f17f0-69fd-49ec-93b0-66333b12198c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:28.735245 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f17f0-69fd-49ec-93b0-66333b12198c admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:28.735867 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f17f0-69fd-49ec-93b0-66333b12198c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:28.737926 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f17f0-69fd-49ec-93b0-66333b12198c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:28.784377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b88db-a284-4fb7-b4e3-b11c189f4aaa None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:28.784502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-543b88db-a284-4fb7-b4e3-b11c189f4aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:28.845485 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 915/1804] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:28 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:25:28.846574 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f17f0-69fd-49ec-93b0-66333b12198c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:28.851707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b88db-a284-4fb7-b4e3-b11c189f4aaa None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:28.920941 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e65ff-f8bf-45b5-843e-b1b9cd8a2f17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:28.921738 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e65ff-f8bf-45b5-843e-b1b9cd8a2f17 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:28.922028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e65ff-f8bf-45b5-843e-b1b9cd8a2f17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:28.922313 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e65ff-f8bf-45b5-843e-b1b9cd8a2f17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:28.944254 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f17f0-69fd-49ec-93b0-66333b12198c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:28.950214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e8380-5ee1-40b1-aea3-f9228073e57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:28.950773 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e8380-5ee1-40b1-aea3-f9228073e57b admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:28.951079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e8380-5ee1-40b1-aea3-f9228073e57b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:28.951409 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e8380-5ee1-40b1-aea3-f9228073e57b admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:28.952070 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e8380-5ee1-40b1-aea3-f9228073e57b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:28.954018 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e8380-5ee1-40b1-aea3-f9228073e57b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:29.036624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e65ff-f8bf-45b5-843e-b1b9cd8a2f17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:29.037940 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e65ff-f8bf-45b5-843e-b1b9cd8a2f17 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:29.047943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 890/1805] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:28 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:25:29.151387 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 916/1806] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:28 2022] POST /identity/v3/users => generated 288 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:25:29.155941 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e65ff-f8bf-45b5-843e-b1b9cd8a2f17 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:29.167147 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e65ff-f8bf-45b5-843e-b1b9cd8a2f17 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:29.172073 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00b8cb-955f-4398-870d-9894340d66e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:29.172690 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00b8cb-955f-4398-870d-9894340d66e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:29.172976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00b8cb-955f-4398-870d-9894340d66e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:29.173370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00b8cb-955f-4398-870d-9894340d66e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:29.174016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00b8cb-955f-4398-870d-9894340d66e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:29.175663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00b8cb-955f-4398-870d-9894340d66e5 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:29.192322 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 891/1807] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:25:29.196267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e8380-5ee1-40b1-aea3-f9228073e57b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:29.206917 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e8380-5ee1-40b1-aea3-f9228073e57b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:29.212095 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b642c-8793-4311-a868-974acb120bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:29.212716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b642c-8793-4311-a868-974acb120bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:29.217487 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b642c-8793-4311-a868-974acb120bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:29.217487 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b642c-8793-4311-a868-974acb120bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:29.217487 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b642c-8793-4311-a868-974acb120bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:29.217487 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b642c-8793-4311-a868-974acb120bb3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:29.226090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 917/1808] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:25:29.230447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00b8cb-955f-4398-870d-9894340d66e5 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:29.241018 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00b8cb-955f-4398-870d-9894340d66e5 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:29.251845 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d5418-518e-4e77-99e8-f229e1fffb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:29.251845 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d5418-518e-4e77-99e8-f229e1fffb3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:29.251845 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d5418-518e-4e77-99e8-f229e1fffb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:29.251845 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d5418-518e-4e77-99e8-f229e1fffb3e admin admin] PATH_INFO: `/v3/projects/ff457732c9fc42ef9c7da7b2c15155f7/users/972c5b4258504adc8bca9cccc0376f91/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:29.264138 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d5418-518e-4e77-99e8-f229e1fffb3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff457732c9fc42ef9c7da7b2c15155f7, user_id=972c5b4258504adc8bca9cccc0376f91, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:29.266856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d5418-518e-4e77-99e8-f229e1fffb3e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:29.281338 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:29.396820 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 892/1809] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:29 2022] PUT /identity/v3/projects/ff457732c9fc42ef9c7da7b2c15155f7/users/972c5b4258504adc8bca9cccc0376f91/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:29.408144 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedaaf7-bb04-4921-a231-4167d7ce3484 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:29.408437 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedaaf7-bb04-4921-a231-4167d7ce3484 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:29.408714 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedaaf7-bb04-4921-a231-4167d7ce3484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:29.442188 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-daedaaf7-bb04-4921-a231-4167d7ce3484 None None] MFA Rules not processed for user `972c5b4258504adc8bca9cccc0376f91`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:29.442188 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-daedaaf7-bb04-4921-a231-4167d7ce3484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:29.458141 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d5418-518e-4e77-99e8-f229e1fffb3e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:29.512866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 918/1810] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:29.520268 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedaaf7-bb04-4921-a231-4167d7ce3484 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:29.520407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d5418-518e-4e77-99e8-f229e1fffb3e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:29.522592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa215bb7-fa9d-4ef9-96e9-353931d2333e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:29.523343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa215bb7-fa9d-4ef9-96e9-353931d2333e service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:29.523699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa215bb7-fa9d-4ef9-96e9-353931d2333e service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:29.524034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa215bb7-fa9d-4ef9-96e9-353931d2333e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:29.525966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-fa215bb7-fa9d-4ef9-96e9-353931d2333e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:29.586479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedaaf7-bb04-4921-a231-4167d7ce3484 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:29.593075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b6c19-e1ba-4732-b9e6-eb0337222a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:29.593824 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b6c19-e1ba-4732-b9e6-eb0337222a6d admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:29.594182 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b6c19-e1ba-4732-b9e6-eb0337222a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:29.594745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b6c19-e1ba-4732-b9e6-eb0337222a6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:29.595792 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b6c19-e1ba-4732-b9e6-eb0337222a6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:29.598115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b6c19-e1ba-4732-b9e6-eb0337222a6d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:29.611428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa215bb7-fa9d-4ef9-96e9-353931d2333e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:29.613329 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa215bb7-fa9d-4ef9-96e9-353931d2333e service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:29.625376 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 893/1811] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:25:29.711919 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 919/1812] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:29 2022] POST /identity/v3/projects => generated 377 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:25:29.716549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa215bb7-fa9d-4ef9-96e9-353931d2333e service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:29.727729 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:29.729797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa215bb7-fa9d-4ef9-96e9-353931d2333e service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:29.785969 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa6d05-0481-4e2b-8169-a1db7bc4bd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:29.786583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa6d05-0481-4e2b-8169-a1db7bc4bd1e admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:29.786892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa6d05-0481-4e2b-8169-a1db7bc4bd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:29.787130 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa6d05-0481-4e2b-8169-a1db7bc4bd1e admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:29.787805 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa6d05-0481-4e2b-8169-a1db7bc4bd1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:29.789803 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa6d05-0481-4e2b-8169-a1db7bc4bd1e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:29.855952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b6c19-e1ba-4732-b9e6-eb0337222a6d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:29.857654 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-472b6c19-e1ba-4732-b9e6-eb0337222a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:29.899206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 894/1813] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:29 2022] POST /identity/v3/users => generated 295 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:29.903359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa6d05-0481-4e2b-8169-a1db7bc4bd1e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:29.913518 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa6d05-0481-4e2b-8169-a1db7bc4bd1e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:29.919562 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a399f0e-d318-4214-9d80-355f47466666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:29.922253 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a399f0e-d318-4214-9d80-355f47466666 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:29.922253 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a399f0e-d318-4214-9d80-355f47466666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:29.922253 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a399f0e-d318-4214-9d80-355f47466666 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:29.922253 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a399f0e-d318-4214-9d80-355f47466666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:29.922992 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a399f0e-d318-4214-9d80-355f47466666 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:29.944626 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 895/1814] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:25:29.944837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b6c19-e1ba-4732-b9e6-eb0337222a6d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:29.945802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a399f0e-d318-4214-9d80-355f47466666 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:29.947237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ba178-2790-41d7-96f1-7ecb74d70739 tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3e5dc6cd39c4af5bc6da31041fc8312', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cffce302cfc34318b20ff6932085637c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:29.948433 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ba178-2790-41d7-96f1-7ecb74d70739 tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:29.948719 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ba178-2790-41d7-96f1-7ecb74d70739 tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:29.948995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ba178-2790-41d7-96f1-7ecb74d70739 tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:29.950602 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ba178-2790-41d7-96f1-7ecb74d70739 tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:29.950899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ba178-2790-41d7-96f1-7ecb74d70739 tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:29.952696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ba178-2790-41d7-96f1-7ecb74d70739 tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:29.956709 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a399f0e-d318-4214-9d80-355f47466666 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:29.961435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29474abe-b1a5-47f3-90f0-3dfc89f5e02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:29.962095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29474abe-b1a5-47f3-90f0-3dfc89f5e02a admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:29.962382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29474abe-b1a5-47f3-90f0-3dfc89f5e02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:29.962658 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29474abe-b1a5-47f3-90f0-3dfc89f5e02a admin admin] PATH_INFO: `/v3/projects/3bb6b5e60c9b4d08a2ce73c8c495c739/users/75478d59dc58409599fd2da9ae0d55ff/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:29.965107 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 920/1815] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:25:29.969427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ba178-2790-41d7-96f1-7ecb74d70739 tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:29.979005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29474abe-b1a5-47f3-90f0-3dfc89f5e02a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bb6b5e60c9b4d08a2ce73c8c495c739, user_id=75478d59dc58409599fd2da9ae0d55ff, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:29.980223 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ba178-2790-41d7-96f1-7ecb74d70739 tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:29.981724 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29474abe-b1a5-47f3-90f0-3dfc89f5e02a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:29.985057 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40986436-dd35-4c63-a360-681cc4763275 tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3e5dc6cd39c4af5bc6da31041fc8312', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cffce302cfc34318b20ff6932085637c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:29.985702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40986436-dd35-4c63-a360-681cc4763275 tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:29.986008 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40986436-dd35-4c63-a360-681cc4763275 tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:29.986293 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40986436-dd35-4c63-a360-681cc4763275 tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:29.986936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40986436-dd35-4c63-a360-681cc4763275 tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:29.988902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40986436-dd35-4c63-a360-681cc4763275 tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:30.234561 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 896/1816] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:29 2022] PUT /identity/v3/projects/3bb6b5e60c9b4d08a2ce73c8c495c739/users/75478d59dc58409599fd2da9ae0d55ff/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:30.238810 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29474abe-b1a5-47f3-90f0-3dfc89f5e02a admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:30.292692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 921/1817] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:29 2022] POST /identity/v3/projects => generated 354 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:25:30.292692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40986436-dd35-4c63-a360-681cc4763275 tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:30.350942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29474abe-b1a5-47f3-90f0-3dfc89f5e02a admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:30.413604 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40986436-dd35-4c63-a360-681cc4763275 tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:30.419631 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74f241-7820-42d8-8765-a8e06429e3ac tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3e5dc6cd39c4af5bc6da31041fc8312', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cffce302cfc34318b20ff6932085637c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:30.420220 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74f241-7820-42d8-8765-a8e06429e3ac tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:30.420505 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74f241-7820-42d8-8765-a8e06429e3ac tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:30.420836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74f241-7820-42d8-8765-a8e06429e3ac tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:30.421627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74f241-7820-42d8-8765-a8e06429e3ac tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:30.421941 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74f241-7820-42d8-8765-a8e06429e3ac tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:30.423679 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74f241-7820-42d8-8765-a8e06429e3ac tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:30.436386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 922/1818] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:25:30.436781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e799d08-0f66-4de2-aea8-f05c7f603601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:30.437227 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74f241-7820-42d8-8765-a8e06429e3ac tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:30.437403 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e799d08-0f66-4de2-aea8-f05c7f603601 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:30.437685 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e799d08-0f66-4de2-aea8-f05c7f603601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:30.438002 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e799d08-0f66-4de2-aea8-f05c7f603601 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:30.438596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e799d08-0f66-4de2-aea8-f05c7f603601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:30.438915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bd74f241-7820-42d8-8765-a8e06429e3ac tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:30.443904 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e799d08-0f66-4de2-aea8-f05c7f603601 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:30.453940 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74f241-7820-42d8-8765-a8e06429e3ac tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:30.457077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 897/1819] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:25:30.459404 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f958186-f2ef-4aa1-a0e4-357c7f95b05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:30.460049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f958186-f2ef-4aa1-a0e4-357c7f95b05f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:30.460346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f958186-f2ef-4aa1-a0e4-357c7f95b05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:30.460623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f958186-f2ef-4aa1-a0e4-357c7f95b05f admin admin] PATH_INFO: `/v3/users/348a742d8d064e60b9112e855a990ea3` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:30.461869 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f958186-f2ef-4aa1-a0e4-357c7f95b05f admin admin] RBAC: Authorizing `identity:delete_user(user_id=348a742d8d064e60b9112e855a990ea3)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:30.462706 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e799d08-0f66-4de2-aea8-f05c7f603601 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:30.463913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f958186-f2ef-4aa1-a0e4-357c7f95b05f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:30.476844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e799d08-0f66-4de2-aea8-f05c7f603601 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:30.482096 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd67ec-f6f0-4ee2-a7f0-c468bb544848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:30.482763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd67ec-f6f0-4ee2-a7f0-c468bb544848 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:30.483043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd67ec-f6f0-4ee2-a7f0-c468bb544848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:30.483350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd67ec-f6f0-4ee2-a7f0-c468bb544848 admin admin] PATH_INFO: `/v3/projects/3bb6b5e60c9b4d08a2ce73c8c495c739/users/75478d59dc58409599fd2da9ae0d55ff/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:30.485291 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd67ec-f6f0-4ee2-a7f0-c468bb544848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bb6b5e60c9b4d08a2ce73c8c495c739, user_id=75478d59dc58409599fd2da9ae0d55ff, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:30.488023 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd67ec-f6f0-4ee2-a7f0-c468bb544848 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:30.499608 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 898/1820] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:30 2022] PUT /identity/v3/projects/3bb6b5e60c9b4d08a2ce73c8c495c739/users/75478d59dc58409599fd2da9ae0d55ff/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:25:30.500498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd67ec-f6f0-4ee2-a7f0-c468bb544848 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:30.562412 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd67ec-f6f0-4ee2-a7f0-c468bb544848 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:30.567302 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0256b9e-4eb3-4023-b91d-b6225f9c7a5e tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3e5dc6cd39c4af5bc6da31041fc8312', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cffce302cfc34318b20ff6932085637c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:30.567978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0256b9e-4eb3-4023-b91d-b6225f9c7a5e tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:30.568406 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0256b9e-4eb3-4023-b91d-b6225f9c7a5e tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:30.568682 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0256b9e-4eb3-4023-b91d-b6225f9c7a5e tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] PATH_INFO: `/v3/projects/5d1c6050e1684c74ba43198e0a827fd0` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:30.569827 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0256b9e-4eb3-4023-b91d-b6225f9c7a5e tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5d1c6050e1684c74ba43198e0a827fd0)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:30.571845 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0256b9e-4eb3-4023-b91d-b6225f9c7a5e tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:30.750352 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9f958186-f2ef-4aa1-a0e4-357c7f95b05f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '348a742d8d064e60b9112e855a990ea3'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:30.771554 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-d0256b9e-4eb3-4023-b91d-b6225f9c7a5e tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d1c6050e1684c74ba43198e0a827fd0', 'request_id': 'req-d0256b9e-4eb3-4023-b91d-b6225f9c7a5e'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:30.781601 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-d0256b9e-4eb3-4023-b91d-b6225f9c7a5e tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d1c6050e1684c74ba43198e0a827fd0', 'request_id': 'req-d0256b9e-4eb3-4023-b91d-b6225f9c7a5e'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:30.781856 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9f958186-f2ef-4aa1-a0e4-357c7f95b05f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '348a742d8d064e60b9112e855a990ea3'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:30.800459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9f958186-f2ef-4aa1-a0e4-357c7f95b05f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '348a742d8d064e60b9112e855a990ea3'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:30.937567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-d0256b9e-4eb3-4023-b91d-b6225f9c7a5e tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d1c6050e1684c74ba43198e0a827fd0', 'request_id': 'req-d0256b9e-4eb3-4023-b91d-b6225f9c7a5e'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:30.954017 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-d0256b9e-4eb3-4023-b91d-b6225f9c7a5e tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] The token cache is being invalidate because project 5d1c6050e1684c74ba43198e0a827fd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:30.954390 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-d0256b9e-4eb3-4023-b91d-b6225f9c7a5e tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:30.955217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-d0256b9e-4eb3-4023-b91d-b6225f9c7a5e tempest-VolumeQuotaClassesTest-1748199210 tempest-VolumeQuotaClassesTest-1748199210-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:30.957293 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 899/1821] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:30 2022] DELETE /identity/v3/projects/5d1c6050e1684c74ba43198e0a827fd0 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:30.958529 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c04eb8-163b-48ac-adcd-f656f3dfdfc1 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:30.958923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c04eb8-163b-48ac-adcd-f656f3dfdfc1 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:30.959332 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c04eb8-163b-48ac-adcd-f656f3dfdfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:31.022440 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-03c04eb8-163b-48ac-adcd-f656f3dfdfc1 None None] MFA Rules not processed for user `75478d59dc58409599fd2da9ae0d55ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:31.023716 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-03c04eb8-163b-48ac-adcd-f656f3dfdfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:31.075864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9f958186-f2ef-4aa1-a0e4-357c7f95b05f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '348a742d8d064e60b9112e855a990ea3'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:31.077414 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9f958186-f2ef-4aa1-a0e4-357c7f95b05f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '348a742d8d064e60b9112e855a990ea3'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:31.086527 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 923/1822] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:30 2022] DELETE /identity/v3/users/348a742d8d064e60b9112e855a990ea3 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:31.091447 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd31131-2a90-49c6-b36c-428028f43f47 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:31.093001 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd31131-2a90-49c6-b36c-428028f43f47 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:31.093001 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd31131-2a90-49c6-b36c-428028f43f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:31.124169 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 900/1823] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:31.139657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c04eb8-163b-48ac-adcd-f656f3dfdfc1 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:31.140817 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-5bd31131-2a90-49c6-b36c-428028f43f47 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:31.141072 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-03c04eb8-163b-48ac-adcd-f656f3dfdfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:31.142498 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5bd31131-2a90-49c6-b36c-428028f43f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:31.209289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c04eb8-163b-48ac-adcd-f656f3dfdfc1 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:31.222806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 924/1824] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:31.235243 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd31131-2a90-49c6-b36c-428028f43f47 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:31.236820 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5bd31131-2a90-49c6-b36c-428028f43f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:31.278598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1106f-0d83-47ec-b418-3cbfa49ae7aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:31.279325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1106f-0d83-47ec-b418-3cbfa49ae7aa service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:31.279614 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1106f-0d83-47ec-b418-3cbfa49ae7aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:31.280197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1106f-0d83-47ec-b418-3cbfa49ae7aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:31.285093 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f7f1106f-0d83-47ec-b418-3cbfa49ae7aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:31.310715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd31131-2a90-49c6-b36c-428028f43f47 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:31.314948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906c49d-94a6-4f81-93b2-61f5629dc749 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:31.315622 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906c49d-94a6-4f81-93b2-61f5629dc749 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:31.315910 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906c49d-94a6-4f81-93b2-61f5629dc749 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:31.316257 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906c49d-94a6-4f81-93b2-61f5629dc749 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:31.333316 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906c49d-94a6-4f81-93b2-61f5629dc749 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:31.335352 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906c49d-94a6-4f81-93b2-61f5629dc749 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:31.349833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 925/1825] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:25:31.388896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1106f-0d83-47ec-b418-3cbfa49ae7aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:31.393177 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1106f-0d83-47ec-b418-3cbfa49ae7aa service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:31.411153 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 901/1826] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:25:31.977943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906c49d-94a6-4f81-93b2-61f5629dc749 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:31.979683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b906c49d-94a6-4f81-93b2-61f5629dc749 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:31.999836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906c49d-94a6-4f81-93b2-61f5629dc749 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:32.014686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fff49-027c-4e61-b562-2f41856c5d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:32.015574 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fff49-027c-4e61-b562-2f41856c5d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:32.016007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fff49-027c-4e61-b562-2f41856c5d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:32.016428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fff49-027c-4e61-b562-2f41856c5d28 admin admin] PATH_INFO: `/v3/projects/be855ec94d234ccab323c123b8ed0aa7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:32.019993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fff49-027c-4e61-b562-2f41856c5d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be855ec94d234ccab323c123b8ed0aa7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:32.019993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fff49-027c-4e61-b562-2f41856c5d28 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:32.147369 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1106f-0d83-47ec-b418-3cbfa49ae7aa service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:32.148918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f7f1106f-0d83-47ec-b418-3cbfa49ae7aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:32.167719 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1106f-0d83-47ec-b418-3cbfa49ae7aa service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:32.173816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d817ea7-e409-4435-8c19-409dca1e6319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:32.180389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d817ea7-e409-4435-8c19-409dca1e6319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:32.180389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d817ea7-e409-4435-8c19-409dca1e6319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:32.180389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d817ea7-e409-4435-8c19-409dca1e6319 admin admin] PATH_INFO: `/v3/users/6936cac7122343938b41c19c961850a5` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:32.188143 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d817ea7-e409-4435-8c19-409dca1e6319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6936cac7122343938b41c19c961850a5)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:32.190038 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d817ea7-e409-4435-8c19-409dca1e6319 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:32.228799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-021fff49-027c-4e61-b562-2f41856c5d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be855ec94d234ccab323c123b8ed0aa7', 'request_id': 'req-021fff49-027c-4e61-b562-2f41856c5d28'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.322092 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-021fff49-027c-4e61-b562-2f41856c5d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be855ec94d234ccab323c123b8ed0aa7', 'request_id': 'req-021fff49-027c-4e61-b562-2f41856c5d28'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.332346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-021fff49-027c-4e61-b562-2f41856c5d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be855ec94d234ccab323c123b8ed0aa7', 'request_id': 'req-021fff49-027c-4e61-b562-2f41856c5d28'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.344116 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-021fff49-027c-4e61-b562-2f41856c5d28 admin admin] The token cache is being invalidate because project be855ec94d234ccab323c123b8ed0aa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:32.344960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-021fff49-027c-4e61-b562-2f41856c5d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.345512 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-021fff49-027c-4e61-b562-2f41856c5d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.355219 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 926/1827] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:31 2022] DELETE /identity/v3/projects/be855ec94d234ccab323c123b8ed0aa7 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:32.356811 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fff49-027c-4e61-b562-2f41856c5d28 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:32.359120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-021fff49-027c-4e61-b562-2f41856c5d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:32.445195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fff49-027c-4e61-b562-2f41856c5d28 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:32.450462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5b775-8222-4dd2-bff4-9becb92d8db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:32.451099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5b775-8222-4dd2-bff4-9becb92d8db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:32.451428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5b775-8222-4dd2-bff4-9becb92d8db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:32.451703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5b775-8222-4dd2-bff4-9becb92d8db9 admin admin] PATH_INFO: `/v3/users/b91aaad8479b4c189ee0d43fc2db6dee` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:32.452843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5b775-8222-4dd2-bff4-9becb92d8db9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b91aaad8479b4c189ee0d43fc2db6dee)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:32.454764 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5b775-8222-4dd2-bff4-9becb92d8db9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:32.592110 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-0d817ea7-e409-4435-8c19-409dca1e6319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6936cac7122343938b41c19c961850a5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.618524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-0d817ea7-e409-4435-8c19-409dca1e6319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6936cac7122343938b41c19c961850a5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.629427 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-0d817ea7-e409-4435-8c19-409dca1e6319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6936cac7122343938b41c19c961850a5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.732921 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-0d817ea7-e409-4435-8c19-409dca1e6319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6936cac7122343938b41c19c961850a5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.733628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-0d817ea7-e409-4435-8c19-409dca1e6319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6936cac7122343938b41c19c961850a5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.746505 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 902/1828] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:32 2022] DELETE /identity/v3/users/6936cac7122343938b41c19c961850a5 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:32.751443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc830c5e-651b-42f4-92f6-aaacc8ca8437 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:32.752052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc830c5e-651b-42f4-92f6-aaacc8ca8437 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:32.752052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc830c5e-651b-42f4-92f6-aaacc8ca8437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:32.758431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-07f5b775-8222-4dd2-bff4-9becb92d8db9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b91aaad8479b4c189ee0d43fc2db6dee'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.798183 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-07f5b775-8222-4dd2-bff4-9becb92d8db9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b91aaad8479b4c189ee0d43fc2db6dee'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.804466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-bc830c5e-651b-42f4-92f6-aaacc8ca8437 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:32.808048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-bc830c5e-651b-42f4-92f6-aaacc8ca8437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:32.820701 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-07f5b775-8222-4dd2-bff4-9becb92d8db9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b91aaad8479b4c189ee0d43fc2db6dee'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.894633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 903/1829] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:32.905001 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc830c5e-651b-42f4-92f6-aaacc8ca8437 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:32.906601 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-bc830c5e-651b-42f4-92f6-aaacc8ca8437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:32.921307 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-07f5b775-8222-4dd2-bff4-9becb92d8db9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b91aaad8479b4c189ee0d43fc2db6dee'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.921886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-07f5b775-8222-4dd2-bff4-9becb92d8db9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b91aaad8479b4c189ee0d43fc2db6dee'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.931637 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 927/1830] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:32 2022] DELETE /identity/v3/users/b91aaad8479b4c189ee0d43fc2db6dee => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:32.977391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc830c5e-651b-42f4-92f6-aaacc8ca8437 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:33.050237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d5d64-3570-4087-96e7-16cc03b751a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:33.050707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d5d64-3570-4087-96e7-16cc03b751a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:33.050995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d5d64-3570-4087-96e7-16cc03b751a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:33.051300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d5d64-3570-4087-96e7-16cc03b751a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:33.053057 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-950d5d64-3570-4087-96e7-16cc03b751a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:33.155391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d5d64-3570-4087-96e7-16cc03b751a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:33.157035 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d5d64-3570-4087-96e7-16cc03b751a5 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:33.169374 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 904/1831] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:25:33.687142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:33.714477 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5b775-8222-4dd2-bff4-9becb92d8db9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:33.716622 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-07f5b775-8222-4dd2-bff4-9becb92d8db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:33.739910 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5b775-8222-4dd2-bff4-9becb92d8db9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:33.751714 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be533f8f-b85c-4c21-859e-cc5d78c366e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:33.752499 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be533f8f-b85c-4c21-859e-cc5d78c366e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:33.752949 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be533f8f-b85c-4c21-859e-cc5d78c366e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:33.753521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be533f8f-b85c-4c21-859e-cc5d78c366e6 admin admin] PATH_INFO: `/v3/projects/0a0f3fb845c04fe98e7c76352902436a` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:33.755138 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be533f8f-b85c-4c21-859e-cc5d78c366e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a0f3fb845c04fe98e7c76352902436a)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:33.757933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be533f8f-b85c-4c21-859e-cc5d78c366e6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:33.863551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83097e01-70e7-448f-9f56-11bc8782fe9b None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:33.864092 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83097e01-70e7-448f-9f56-11bc8782fe9b None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:33.864169 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83097e01-70e7-448f-9f56-11bc8782fe9b None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:33.865608 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 905/1832] 10.222.0.78 () {54 vars in 921 bytes} [Mon Sep 19 11:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:25:33.891742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876f809-7dc5-4a7b-9427-19843318245b None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:33.892611 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876f809-7dc5-4a7b-9427-19843318245b None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:33.902927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876f809-7dc5-4a7b-9427-19843318245b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:33.942991 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-be533f8f-b85c-4c21-859e-cc5d78c366e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a0f3fb845c04fe98e7c76352902436a', 'request_id': 'req-be533f8f-b85c-4c21-859e-cc5d78c366e6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:33.951443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-d876f809-7dc5-4a7b-9427-19843318245b None None] MFA Rules not processed for user `aba1968529574fe5b4c5d114c874f428`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:33.953607 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d876f809-7dc5-4a7b-9427-19843318245b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:34.018470 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-be533f8f-b85c-4c21-859e-cc5d78c366e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a0f3fb845c04fe98e7c76352902436a', 'request_id': 'req-be533f8f-b85c-4c21-859e-cc5d78c366e6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:34.028794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-be533f8f-b85c-4c21-859e-cc5d78c366e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a0f3fb845c04fe98e7c76352902436a', 'request_id': 'req-be533f8f-b85c-4c21-859e-cc5d78c366e6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:34.046525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-be533f8f-b85c-4c21-859e-cc5d78c366e6 admin admin] The token cache is being invalidate because project 0a0f3fb845c04fe98e7c76352902436a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:34.047583 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-be533f8f-b85c-4c21-859e-cc5d78c366e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:34.048826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-be533f8f-b85c-4c21-859e-cc5d78c366e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:34.056730 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 928/1833] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:33 2022] DELETE /identity/v3/projects/0a0f3fb845c04fe98e7c76352902436a => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:34.059398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c66586-e4c4-4be0-9fb8-0177b8c32c5d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:34.063573 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c66586-e4c4-4be0-9fb8-0177b8c32c5d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:34.064037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c66586-e4c4-4be0-9fb8-0177b8c32c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:34.122169 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:34.125281 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c2c66586-e4c4-4be0-9fb8-0177b8c32c5d None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:34.131202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c2c66586-e4c4-4be0-9fb8-0177b8c32c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:34.154765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 906/1834] 10.222.0.78 () {58 vars in 1018 bytes} [Mon Sep 19 11:25:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:34.155698 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876f809-7dc5-4a7b-9427-19843318245b None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:34.168427 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d876f809-7dc5-4a7b-9427-19843318245b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:34.258864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 929/1835] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:34.258864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c66586-e4c4-4be0-9fb8-0177b8c32c5d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:34.258864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c2c66586-e4c4-4be0-9fb8-0177b8c32c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:34.277146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c66586-e4c4-4be0-9fb8-0177b8c32c5d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:34.288947 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6dfc8-b466-4c58-8635-f9074cc6c4a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:34.289593 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6dfc8-b466-4c58-8635-f9074cc6c4a9 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:34.289884 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6dfc8-b466-4c58-8635-f9074cc6c4a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:34.290167 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6dfc8-b466-4c58-8635-f9074cc6c4a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:34.298726 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876f809-7dc5-4a7b-9427-19843318245b None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:34.303884 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6dfc8-b466-4c58-8635-f9074cc6c4a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:34.304225 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cab585-8271-4db9-b69b-7d859105265f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:34.304866 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cab585-8271-4db9-b69b-7d859105265f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:34.305152 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cab585-8271-4db9-b69b-7d859105265f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:34.305429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cab585-8271-4db9-b69b-7d859105265f admin admin] PATH_INFO: `/v3/projects/3bead1f8c4e64e128d92f2c734d456c5` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:34.305534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6dfc8-b466-4c58-8635-f9074cc6c4a9 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:34.306784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cab585-8271-4db9-b69b-7d859105265f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bead1f8c4e64e128d92f2c734d456c5)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:34.309232 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cab585-8271-4db9-b69b-7d859105265f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:34.317158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 930/1836] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:34.318432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0729743-19d9-4a0d-a877-961c44b8be91 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:34.318717 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0729743-19d9-4a0d-a877-961c44b8be91 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:34.319322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0729743-19d9-4a0d-a877-961c44b8be91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:34.342139 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f0729743-19d9-4a0d-a877-961c44b8be91 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:34.343607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f0729743-19d9-4a0d-a877-961c44b8be91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:34.355073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 931/1837] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:34.359461 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0729743-19d9-4a0d-a877-961c44b8be91 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:34.372354 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0729743-19d9-4a0d-a877-961c44b8be91 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:34.378229 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b7d34-7a78-4f0b-88f4-179939d2377f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:34.379365 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b7d34-7a78-4f0b-88f4-179939d2377f admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:34.379801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b7d34-7a78-4f0b-88f4-179939d2377f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:34.380222 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b7d34-7a78-4f0b-88f4-179939d2377f admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:34.381203 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b7d34-7a78-4f0b-88f4-179939d2377f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:34.381472 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b7d34-7a78-4f0b-88f4-179939d2377f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:34.383329 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b7d34-7a78-4f0b-88f4-179939d2377f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:34.420736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 932/1838] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:25:34.425001 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b7d34-7a78-4f0b-88f4-179939d2377f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:34.436106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b7d34-7a78-4f0b-88f4-179939d2377f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:34.460848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-b5cab585-8271-4db9-b69b-7d859105265f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bead1f8c4e64e128d92f2c734d456c5', 'request_id': 'req-b5cab585-8271-4db9-b69b-7d859105265f'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:34.466830 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-b5cab585-8271-4db9-b69b-7d859105265f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bead1f8c4e64e128d92f2c734d456c5', 'request_id': 'req-b5cab585-8271-4db9-b69b-7d859105265f'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:34.538011 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d60efe-0c92-4f87-878c-f462f0fd5e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:34.538801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d60efe-0c92-4f87-878c-f462f0fd5e2c admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:34.539415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d60efe-0c92-4f87-878c-f462f0fd5e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:34.539699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d60efe-0c92-4f87-878c-f462f0fd5e2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:34.540393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d60efe-0c92-4f87-878c-f462f0fd5e2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:34.544381 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d60efe-0c92-4f87-878c-f462f0fd5e2c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:34.565484 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-b5cab585-8271-4db9-b69b-7d859105265f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bead1f8c4e64e128d92f2c734d456c5', 'request_id': 'req-b5cab585-8271-4db9-b69b-7d859105265f'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:34.578679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-b5cab585-8271-4db9-b69b-7d859105265f admin admin] The token cache is being invalidate because project 3bead1f8c4e64e128d92f2c734d456c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:34.582932 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-b5cab585-8271-4db9-b69b-7d859105265f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:34.584026 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-b5cab585-8271-4db9-b69b-7d859105265f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:34.590039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 907/1839] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:34 2022] DELETE /identity/v3/projects/3bead1f8c4e64e128d92f2c734d456c5 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:34.596768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cab585-8271-4db9-b69b-7d859105265f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:34.599686 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b5cab585-8271-4db9-b69b-7d859105265f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:34.636301 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cab585-8271-4db9-b69b-7d859105265f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:34.643273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484acc5-4d3f-4970-b718-8b7ea2747fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:34.643939 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484acc5-4d3f-4970-b718-8b7ea2747fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:34.644255 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484acc5-4d3f-4970-b718-8b7ea2747fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:34.644534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484acc5-4d3f-4970-b718-8b7ea2747fd7 admin admin] PATH_INFO: `/v3/users/b3e5dc6cd39c4af5bc6da31041fc8312` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:34.645996 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484acc5-4d3f-4970-b718-8b7ea2747fd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3e5dc6cd39c4af5bc6da31041fc8312)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:34.648301 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484acc5-4d3f-4970-b718-8b7ea2747fd7 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:34.676120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 933/1840] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:34 2022] POST /identity/v3/projects => generated 373 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:34.679535 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d60efe-0c92-4f87-878c-f462f0fd5e2c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:34.681339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b8d60efe-0c92-4f87-878c-f462f0fd5e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:34.764223 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d60efe-0c92-4f87-878c-f462f0fd5e2c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:34.772890 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86091265-0864-4cfb-b19b-e53d059eac54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:34.773524 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86091265-0864-4cfb-b19b-e53d059eac54 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:34.773809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86091265-0864-4cfb-b19b-e53d059eac54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:34.774088 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86091265-0864-4cfb-b19b-e53d059eac54 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:34.774744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86091265-0864-4cfb-b19b-e53d059eac54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:34.777407 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86091265-0864-4cfb-b19b-e53d059eac54 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:34.855678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1484acc5-4d3f-4970-b718-8b7ea2747fd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3e5dc6cd39c4af5bc6da31041fc8312'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:34.867713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1484acc5-4d3f-4970-b718-8b7ea2747fd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3e5dc6cd39c4af5bc6da31041fc8312'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:34.875187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1484acc5-4d3f-4970-b718-8b7ea2747fd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3e5dc6cd39c4af5bc6da31041fc8312'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:34.904525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 934/1841] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:34 2022] POST /identity/v3/users => generated 287 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:34.905290 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86091265-0864-4cfb-b19b-e53d059eac54 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:34.906838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-86091265-0864-4cfb-b19b-e53d059eac54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:34.918785 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86091265-0864-4cfb-b19b-e53d059eac54 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:34.923386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4bc08-f58e-4b02-aa13-6018485a06c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:34.924001 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4bc08-f58e-4b02-aa13-6018485a06c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:34.924280 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4bc08-f58e-4b02-aa13-6018485a06c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:34.924554 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4bc08-f58e-4b02-aa13-6018485a06c0 admin admin] PATH_INFO: `/v3/users/45a6def2b0c442f48af637c0715f3a69` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:34.925850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4bc08-f58e-4b02-aa13-6018485a06c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45a6def2b0c442f48af637c0715f3a69)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:34.927792 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4bc08-f58e-4b02-aa13-6018485a06c0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:34.967072 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1484acc5-4d3f-4970-b718-8b7ea2747fd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3e5dc6cd39c4af5bc6da31041fc8312'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:34.967721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1484acc5-4d3f-4970-b718-8b7ea2747fd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3e5dc6cd39c4af5bc6da31041fc8312'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:34.974529 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 908/1842] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:34 2022] DELETE /identity/v3/users/b3e5dc6cd39c4af5bc6da31041fc8312 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:25:34.975484 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484acc5-4d3f-4970-b718-8b7ea2747fd7 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:34.976918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1484acc5-4d3f-4970-b718-8b7ea2747fd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:35.044762 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484acc5-4d3f-4970-b718-8b7ea2747fd7 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:35.050556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1104c56-1340-41ee-a8cf-7ecb1a69385f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:35.051189 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1104c56-1340-41ee-a8cf-7ecb1a69385f admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:35.053501 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1104c56-1340-41ee-a8cf-7ecb1a69385f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:35.054805 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1104c56-1340-41ee-a8cf-7ecb1a69385f admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:35.055712 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1104c56-1340-41ee-a8cf-7ecb1a69385f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:35.057393 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1104c56-1340-41ee-a8cf-7ecb1a69385f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:35.068092 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 909/1843] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:35.072037 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1104c56-1340-41ee-a8cf-7ecb1a69385f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:35.082300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1104c56-1340-41ee-a8cf-7ecb1a69385f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:35.086870 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19366a9a-6204-4385-8172-7e3c55ab36ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:35.087517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19366a9a-6204-4385-8172-7e3c55ab36ad admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:35.087796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19366a9a-6204-4385-8172-7e3c55ab36ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:35.088683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19366a9a-6204-4385-8172-7e3c55ab36ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:35.089322 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19366a9a-6204-4385-8172-7e3c55ab36ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:35.091446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19366a9a-6204-4385-8172-7e3c55ab36ad admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:35.101365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 910/1844] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:25:35.105585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19366a9a-6204-4385-8172-7e3c55ab36ad admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:35.115616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19366a9a-6204-4385-8172-7e3c55ab36ad admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:35.120521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977f702-19ab-4ee5-9125-cc3b3f567d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:35.121171 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977f702-19ab-4ee5-9125-cc3b3f567d69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:35.121560 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977f702-19ab-4ee5-9125-cc3b3f567d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:35.121836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977f702-19ab-4ee5-9125-cc3b3f567d69 admin admin] PATH_INFO: `/v3/projects/c3d6c73709914b8286ae13a68710fce3/users/c84f30af07de4ba1a99f03cfe56cad45/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:35.138408 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977f702-19ab-4ee5-9125-cc3b3f567d69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d6c73709914b8286ae13a68710fce3, user_id=c84f30af07de4ba1a99f03cfe56cad45, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:35.141304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977f702-19ab-4ee5-9125-cc3b3f567d69 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:35.157370 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cdb4bc08-f58e-4b02-aa13-6018485a06c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45a6def2b0c442f48af637c0715f3a69'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:35.187382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cdb4bc08-f58e-4b02-aa13-6018485a06c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45a6def2b0c442f48af637c0715f3a69'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:35.196352 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cdb4bc08-f58e-4b02-aa13-6018485a06c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45a6def2b0c442f48af637c0715f3a69'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:35.232284 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 911/1845] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:35 2022] PUT /identity/v3/projects/c3d6c73709914b8286ae13a68710fce3/users/c84f30af07de4ba1a99f03cfe56cad45/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:25:35.249555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bd500-727b-40c3-9286-607abbb53a5b None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:35.249555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bd500-727b-40c3-9286-607abbb53a5b None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:35.251391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bd500-727b-40c3-9286-607abbb53a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:35.296119 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cdb4bc08-f58e-4b02-aa13-6018485a06c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45a6def2b0c442f48af637c0715f3a69'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:35.296855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cdb4bc08-f58e-4b02-aa13-6018485a06c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45a6def2b0c442f48af637c0715f3a69'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:35.306012 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-cc6bd500-727b-40c3-9286-607abbb53a5b None None] MFA Rules not processed for user `c84f30af07de4ba1a99f03cfe56cad45`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:35.310246 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 935/1846] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:34 2022] DELETE /identity/v3/users/45a6def2b0c442f48af637c0715f3a69 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:35.313197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-cc6bd500-727b-40c3-9286-607abbb53a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:35.314443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93922479-6989-4f03-b968-35713f92c0c6 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:35.314943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93922479-6989-4f03-b968-35713f92c0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:35.315850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93922479-6989-4f03-b968-35713f92c0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:35.342952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-93922479-6989-4f03-b968-35713f92c0c6 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:35.345097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-93922479-6989-4f03-b968-35713f92c0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:35.424783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 912/1847] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:35.437995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bd500-727b-40c3-9286-607abbb53a5b None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:35.439507 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-cc6bd500-727b-40c3-9286-607abbb53a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:35.454963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 936/1848] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:35.456073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93922479-6989-4f03-b968-35713f92c0c6 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:35.457519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-93922479-6989-4f03-b968-35713f92c0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:35.471710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93922479-6989-4f03-b968-35713f92c0c6 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:35.487685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c69e6-065d-4681-8a1a-1cb090bce4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:35.488427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c69e6-065d-4681-8a1a-1cb090bce4e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:35.488811 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c69e6-065d-4681-8a1a-1cb090bce4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:35.489147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c69e6-065d-4681-8a1a-1cb090bce4e9 admin admin] PATH_INFO: `/v3/projects/cffce302cfc34318b20ff6932085637c` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:35.491051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c69e6-065d-4681-8a1a-1cb090bce4e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cffce302cfc34318b20ff6932085637c)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:35.495714 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c69e6-065d-4681-8a1a-1cb090bce4e9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:35.526515 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bd500-727b-40c3-9286-607abbb53a5b None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:35.530093 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca4e7c-01db-4dd9-a0a7-0dd8f2c00fcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:35.530775 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca4e7c-01db-4dd9-a0a7-0dd8f2c00fcf service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:35.530983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca4e7c-01db-4dd9-a0a7-0dd8f2c00fcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:35.531289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca4e7c-01db-4dd9-a0a7-0dd8f2c00fcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:35.539948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca4e7c-01db-4dd9-a0a7-0dd8f2c00fcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:35.541959 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca4e7c-01db-4dd9-a0a7-0dd8f2c00fcf service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:35.550735 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 913/1849] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:25:35.551553 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca4e7c-01db-4dd9-a0a7-0dd8f2c00fcf service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:35.552808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b1ca4e7c-01db-4dd9-a0a7-0dd8f2c00fcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:35.613734 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca4e7c-01db-4dd9-a0a7-0dd8f2c00fcf service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:35.618167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094a5be-f496-4d5c-a39a-5e3998b6cf63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:35.618771 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094a5be-f496-4d5c-a39a-5e3998b6cf63 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:35.619048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094a5be-f496-4d5c-a39a-5e3998b6cf63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:35.619893 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094a5be-f496-4d5c-a39a-5e3998b6cf63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:35.629534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094a5be-f496-4d5c-a39a-5e3998b6cf63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:35.630957 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094a5be-f496-4d5c-a39a-5e3998b6cf63 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:35.641103 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 914/1850] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:35.644683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b53c69e6-065d-4681-8a1a-1cb090bce4e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cffce302cfc34318b20ff6932085637c', 'request_id': 'req-b53c69e6-065d-4681-8a1a-1cb090bce4e9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:35.715656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b53c69e6-065d-4681-8a1a-1cb090bce4e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cffce302cfc34318b20ff6932085637c', 'request_id': 'req-b53c69e6-065d-4681-8a1a-1cb090bce4e9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:35.723085 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b53c69e6-065d-4681-8a1a-1cb090bce4e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cffce302cfc34318b20ff6932085637c', 'request_id': 'req-b53c69e6-065d-4681-8a1a-1cb090bce4e9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:35.734673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b53c69e6-065d-4681-8a1a-1cb090bce4e9 admin admin] The token cache is being invalidate because project cffce302cfc34318b20ff6932085637c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:35.734968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b53c69e6-065d-4681-8a1a-1cb090bce4e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:35.735679 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b53c69e6-065d-4681-8a1a-1cb090bce4e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:35.737442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 937/1851] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:35 2022] DELETE /identity/v3/projects/cffce302cfc34318b20ff6932085637c => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:35.742725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479c9c3-1bfa-49f5-bc8b-aa4964bf9e77 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:35.743014 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479c9c3-1bfa-49f5-bc8b-aa4964bf9e77 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:35.743335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479c9c3-1bfa-49f5-bc8b-aa4964bf9e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:35.763814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-5479c9c3-1bfa-49f5-bc8b-aa4964bf9e77 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:35.765026 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5479c9c3-1bfa-49f5-bc8b-aa4964bf9e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:35.835488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 915/1852] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:35.843349 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87a042-8f04-4b43-a5f8-f322318225fa None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:35.843931 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87a042-8f04-4b43-a5f8-f322318225fa None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:35.843931 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87a042-8f04-4b43-a5f8-f322318225fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:35.888998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-de87a042-8f04-4b43-a5f8-f322318225fa None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:35.890309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-de87a042-8f04-4b43-a5f8-f322318225fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:35.898579 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 938/1853] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:35.903742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479c9c3-1bfa-49f5-bc8b-aa4964bf9e77 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:35.920777 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479c9c3-1bfa-49f5-bc8b-aa4964bf9e77 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:35.924798 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e6dad-1154-466a-bace-7c6e11b3531a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:35.925472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e6dad-1154-466a-bace-7c6e11b3531a admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:35.925774 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e6dad-1154-466a-bace-7c6e11b3531a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:35.926069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e6dad-1154-466a-bace-7c6e11b3531a admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:35.926732 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e6dad-1154-466a-bace-7c6e11b3531a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:35.928328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e6dad-1154-466a-bace-7c6e11b3531a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:35.929983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e6dad-1154-466a-bace-7c6e11b3531a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:35.947346 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 916/1854] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:25:35.951855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87a042-8f04-4b43-a5f8-f322318225fa None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:35.964180 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87a042-8f04-4b43-a5f8-f322318225fa None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:35.970855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810a60f-a5a1-4a48-b402-171b72fd4e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:35.971695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810a60f-a5a1-4a48-b402-171b72fd4e8e admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:35.972289 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810a60f-a5a1-4a48-b402-171b72fd4e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:35.972712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810a60f-a5a1-4a48-b402-171b72fd4e8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:35.973515 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c810a60f-a5a1-4a48-b402-171b72fd4e8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:35.975627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c810a60f-a5a1-4a48-b402-171b72fd4e8e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:36.084237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 939/1855] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:35 2022] POST /identity/v3/projects => generated 381 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:36.088999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e6dad-1154-466a-bace-7c6e11b3531a admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:36.100102 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e6dad-1154-466a-bace-7c6e11b3531a admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:36.181589 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1493a6-d51e-4970-bd1d-63e63bf9b1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:36.182218 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1493a6-d51e-4970-bd1d-63e63bf9b1d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:36.182485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1493a6-d51e-4970-bd1d-63e63bf9b1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:36.182768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1493a6-d51e-4970-bd1d-63e63bf9b1d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:36.183447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1493a6-d51e-4970-bd1d-63e63bf9b1d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:36.185362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1493a6-d51e-4970-bd1d-63e63bf9b1d5 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:36.227636 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810a60f-a5a1-4a48-b402-171b72fd4e8e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:36.231869 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c810a60f-a5a1-4a48-b402-171b72fd4e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:36.255860 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810a60f-a5a1-4a48-b402-171b72fd4e8e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:36.258285 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b9e16-8620-4b95-8b02-8286d8b53309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:36.259527 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b9e16-8620-4b95-8b02-8286d8b53309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:36.260566 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b9e16-8620-4b95-8b02-8286d8b53309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:36.260852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b9e16-8620-4b95-8b02-8286d8b53309 admin admin] PATH_INFO: `/v3/projects/fced8c972a7d4ee8aa530835a32bc80a` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:36.262212 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b9e16-8620-4b95-8b02-8286d8b53309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fced8c972a7d4ee8aa530835a32bc80a)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:36.265345 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b9e16-8620-4b95-8b02-8286d8b53309 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:36.366005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 917/1856] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:36 2022] POST /identity/v3/users => generated 291 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:36.369969 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1493a6-d51e-4970-bd1d-63e63bf9b1d5 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:36.380183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1493a6-d51e-4970-bd1d-63e63bf9b1d5 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:36.385599 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11cb88-7bc9-474a-a7ef-7bffefb93fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:36.386214 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11cb88-7bc9-474a-a7ef-7bffefb93fac admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:36.386592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11cb88-7bc9-474a-a7ef-7bffefb93fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:36.386882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11cb88-7bc9-474a-a7ef-7bffefb93fac admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:36.387567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11cb88-7bc9-474a-a7ef-7bffefb93fac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:36.389223 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11cb88-7bc9-474a-a7ef-7bffefb93fac admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:36.400010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 918/1857] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:36.404463 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11cb88-7bc9-474a-a7ef-7bffefb93fac admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:36.414872 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11cb88-7bc9-474a-a7ef-7bffefb93fac admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:36.428113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123e82c-15ef-42c6-b836-640c125a1d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:36.428113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123e82c-15ef-42c6-b836-640c125a1d62 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:36.428113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123e82c-15ef-42c6-b836-640c125a1d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:36.428113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123e82c-15ef-42c6-b836-640c125a1d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:36.428411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123e82c-15ef-42c6-b836-640c125a1d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:36.429084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123e82c-15ef-42c6-b836-640c125a1d62 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:36.442402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 919/1858] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:25:36.446262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123e82c-15ef-42c6-b836-640c125a1d62 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:36.448260 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-741b9e16-8620-4b95-8b02-8286d8b53309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fced8c972a7d4ee8aa530835a32bc80a', 'request_id': 'req-741b9e16-8620-4b95-8b02-8286d8b53309'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:36.459023 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123e82c-15ef-42c6-b836-640c125a1d62 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:36.531019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ff46a-0824-4dd3-936f-392f7d48bfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:36.531703 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ff46a-0824-4dd3-936f-392f7d48bfe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:36.531986 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ff46a-0824-4dd3-936f-392f7d48bfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:36.532263 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ff46a-0824-4dd3-936f-392f7d48bfe1 admin admin] PATH_INFO: `/v3/projects/2a214f36bf364c6db3f3507fd93b64f6/users/bc872356cbba46539164ae0944f950ea/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:36.547378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738ff46a-0824-4dd3-936f-392f7d48bfe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a214f36bf364c6db3f3507fd93b64f6, user_id=bc872356cbba46539164ae0944f950ea, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:36.550102 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738ff46a-0824-4dd3-936f-392f7d48bfe1 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:36.606702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-741b9e16-8620-4b95-8b02-8286d8b53309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fced8c972a7d4ee8aa530835a32bc80a', 'request_id': 'req-741b9e16-8620-4b95-8b02-8286d8b53309'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:36.614026 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-741b9e16-8620-4b95-8b02-8286d8b53309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fced8c972a7d4ee8aa530835a32bc80a', 'request_id': 'req-741b9e16-8620-4b95-8b02-8286d8b53309'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:36.625716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-741b9e16-8620-4b95-8b02-8286d8b53309 admin admin] The token cache is being invalidate because project fced8c972a7d4ee8aa530835a32bc80a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:36.626013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-741b9e16-8620-4b95-8b02-8286d8b53309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:36.626697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-741b9e16-8620-4b95-8b02-8286d8b53309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:36.628352 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 940/1859] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:36 2022] DELETE /identity/v3/projects/fced8c972a7d4ee8aa530835a32bc80a => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:36.632845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b9e16-8620-4b95-8b02-8286d8b53309 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:36.634268 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-741b9e16-8620-4b95-8b02-8286d8b53309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:36.643242 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 920/1860] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:36 2022] PUT /identity/v3/projects/2a214f36bf364c6db3f3507fd93b64f6/users/bc872356cbba46539164ae0944f950ea/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:25:36.648552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b9e16-8620-4b95-8b02-8286d8b53309 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:36.670231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba73e5c-56c4-492d-ba9f-877e9c9d4bbd None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:36.670231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba73e5c-56c4-492d-ba9f-877e9c9d4bbd None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:36.670231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba73e5c-56c4-492d-ba9f-877e9c9d4bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:36.687387 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-1ba73e5c-56c4-492d-ba9f-877e9c9d4bbd None None] MFA Rules not processed for user `bc872356cbba46539164ae0944f950ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:36.687387 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1ba73e5c-56c4-492d-ba9f-877e9c9d4bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:36.717672 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76507cbb-8c4f-4584-9ec6-8dba308df401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:36.718327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76507cbb-8c4f-4584-9ec6-8dba308df401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:36.718610 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76507cbb-8c4f-4584-9ec6-8dba308df401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:36.718886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76507cbb-8c4f-4584-9ec6-8dba308df401 admin admin] PATH_INFO: `/v3/users/51e5d7e02f494ef2bb0bb9fa165f895e` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:36.720091 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76507cbb-8c4f-4584-9ec6-8dba308df401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51e5d7e02f494ef2bb0bb9fa165f895e)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:36.722039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76507cbb-8c4f-4584-9ec6-8dba308df401 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:36.752374 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 921/1861] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:36.756914 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba73e5c-56c4-492d-ba9f-877e9c9d4bbd None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:36.758236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1ba73e5c-56c4-492d-ba9f-877e9c9d4bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:36.771557 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba73e5c-56c4-492d-ba9f-877e9c9d4bbd None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:36.772732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:36.776585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf201d7-9691-4005-bc0c-dbed9d8937d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:36.777213 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf201d7-9691-4005-bc0c-dbed9d8937d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:36.777491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf201d7-9691-4005-bc0c-dbed9d8937d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:36.777762 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf201d7-9691-4005-bc0c-dbed9d8937d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:36.778390 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf201d7-9691-4005-bc0c-dbed9d8937d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:36.780321 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf201d7-9691-4005-bc0c-dbed9d8937d9 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:36.892494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 922/1862] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:36 2022] POST /identity/v3/projects => generated 381 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:25:36.893813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf201d7-9691-4005-bc0c-dbed9d8937d9 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:36.948160 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-76507cbb-8c4f-4584-9ec6-8dba308df401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51e5d7e02f494ef2bb0bb9fa165f895e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:36.960410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf201d7-9691-4005-bc0c-dbed9d8937d9 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:36.960897 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-76507cbb-8c4f-4584-9ec6-8dba308df401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51e5d7e02f494ef2bb0bb9fa165f895e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:36.966069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246f2cf-8d32-49d3-bf7b-1f13ce0012bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:36.966691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246f2cf-8d32-49d3-bf7b-1f13ce0012bb admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:36.966969 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246f2cf-8d32-49d3-bf7b-1f13ce0012bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:36.967272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246f2cf-8d32-49d3-bf7b-1f13ce0012bb admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:36.967910 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246f2cf-8d32-49d3-bf7b-1f13ce0012bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:36.968286 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-76507cbb-8c4f-4584-9ec6-8dba308df401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51e5d7e02f494ef2bb0bb9fa165f895e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:36.970004 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246f2cf-8d32-49d3-bf7b-1f13ce0012bb admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:37.075227 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-76507cbb-8c4f-4584-9ec6-8dba308df401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51e5d7e02f494ef2bb0bb9fa165f895e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:37.075862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-76507cbb-8c4f-4584-9ec6-8dba308df401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51e5d7e02f494ef2bb0bb9fa165f895e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:37.082463 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 923/1863] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:36 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:37.084418 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 941/1864] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:36 2022] DELETE /identity/v3/users/51e5d7e02f494ef2bb0bb9fa165f895e => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:37.085788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246f2cf-8d32-49d3-bf7b-1f13ce0012bb admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:37.087558 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d246f2cf-8d32-49d3-bf7b-1f13ce0012bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:37.100407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246f2cf-8d32-49d3-bf7b-1f13ce0012bb admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:37.170894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3216ef3-dabf-448e-a160-4682aa7b995b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:37.171341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3216ef3-dabf-448e-a160-4682aa7b995b admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:37.171627 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3216ef3-dabf-448e-a160-4682aa7b995b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:37.172202 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3216ef3-dabf-448e-a160-4682aa7b995b admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:37.172600 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3216ef3-dabf-448e-a160-4682aa7b995b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:37.174297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3216ef3-dabf-448e-a160-4682aa7b995b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:37.187195 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 924/1865] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:37.192727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:37.194464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76507cbb-8c4f-4584-9ec6-8dba308df401 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:37.208245 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76507cbb-8c4f-4584-9ec6-8dba308df401 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:37.223712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479404ad-e995-4a84-9e09-081fd7a4e05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:37.224401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479404ad-e995-4a84-9e09-081fd7a4e05d admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:37.224681 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479404ad-e995-4a84-9e09-081fd7a4e05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:37.224958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479404ad-e995-4a84-9e09-081fd7a4e05d admin admin] PATH_INFO: `/v3/projects/7ce4a0f15b344cff891072b691030897/users/79fdb72e792648ad9e684403f97b0f0a/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:37.273799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479404ad-e995-4a84-9e09-081fd7a4e05d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ce4a0f15b344cff891072b691030897, user_id=79fdb72e792648ad9e684403f97b0f0a, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:37.279153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479404ad-e995-4a84-9e09-081fd7a4e05d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:37.407187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 942/1866] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:37 2022] PUT /identity/v3/projects/7ce4a0f15b344cff891072b691030897/users/79fdb72e792648ad9e684403f97b0f0a/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:37.408434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3216ef3-dabf-448e-a160-4682aa7b995b admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:37.421956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3216ef3-dabf-448e-a160-4682aa7b995b admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:37.532081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3c54a-0c4d-40cc-a74b-eff6f725bb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:37.533052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3c54a-0c4d-40cc-a74b-eff6f725bb4a admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:37.533484 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3c54a-0c4d-40cc-a74b-eff6f725bb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:37.533941 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3c54a-0c4d-40cc-a74b-eff6f725bb4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:37.535626 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3c54a-0c4d-40cc-a74b-eff6f725bb4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:37.538473 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3c54a-0c4d-40cc-a74b-eff6f725bb4a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:37.559881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 925/1867] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:25:37.563995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479404ad-e995-4a84-9e09-081fd7a4e05d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:37.577888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479404ad-e995-4a84-9e09-081fd7a4e05d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:37.584826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7aed3-ec7a-4f87-af99-ce381741a8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:37.585751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7aed3-ec7a-4f87-af99-ce381741a8c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:37.586216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7aed3-ec7a-4f87-af99-ce381741a8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:37.586699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7aed3-ec7a-4f87-af99-ce381741a8c4 admin admin] PATH_INFO: `/v3/projects/7ce4a0f15b344cff891072b691030897/users/79fdb72e792648ad9e684403f97b0f0a/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:37.589685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7aed3-ec7a-4f87-af99-ce381741a8c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ce4a0f15b344cff891072b691030897, user_id=79fdb72e792648ad9e684403f97b0f0a, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:37.592936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7aed3-ec7a-4f87-af99-ce381741a8c4 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:37.606834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 943/1868] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:37 2022] PUT /identity/v3/projects/7ce4a0f15b344cff891072b691030897/users/79fdb72e792648ad9e684403f97b0f0a/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:25:37.616903 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ac3e4-efc0-4c9e-a295-01ea0243939e None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:37.617570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ac3e4-efc0-4c9e-a295-01ea0243939e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:37.618000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ac3e4-efc0-4c9e-a295-01ea0243939e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:37.653086 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-a58ac3e4-efc0-4c9e-a295-01ea0243939e None None] MFA Rules not processed for user `79fdb72e792648ad9e684403f97b0f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:37.654677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a58ac3e4-efc0-4c9e-a295-01ea0243939e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:37.751226 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 926/1869] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:37 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:37.763335 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7aed3-ec7a-4f87-af99-ce381741a8c4 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:37.765558 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d0d7aed3-ec7a-4f87-af99-ce381741a8c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:37.855864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7aed3-ec7a-4f87-af99-ce381741a8c4 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:37.860296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a4c7b-42bb-442c-833b-697beadaf1ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:37.860856 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a4c7b-42bb-442c-833b-697beadaf1ad service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:37.861140 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a4c7b-42bb-442c-833b-697beadaf1ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:37.861412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a4c7b-42bb-442c-833b-697beadaf1ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:37.871455 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a4c7b-42bb-442c-833b-697beadaf1ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:37.872902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a4c7b-42bb-442c-833b-697beadaf1ad service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:37.882831 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 944/1870] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:37 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:38.007043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ac3e4-efc0-4c9e-a295-01ea0243939e None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:38.016789 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ac3e4-efc0-4c9e-a295-01ea0243939e None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:38.020346 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8ca60-2478-45bc-a2b7-39b045cad2d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:38.020946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8ca60-2478-45bc-a2b7-39b045cad2d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:38.021220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8ca60-2478-45bc-a2b7-39b045cad2d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:38.021513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8ca60-2478-45bc-a2b7-39b045cad2d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:38.023980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-51f8ca60-2478-45bc-a2b7-39b045cad2d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:38.085524 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a4c7b-42bb-442c-833b-697beadaf1ad service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:38.085524 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0a5a4c7b-42bb-442c-833b-697beadaf1ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:38.086041 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8ca60-2478-45bc-a2b7-39b045cad2d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:38.087731 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8ca60-2478-45bc-a2b7-39b045cad2d3 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:38.099361 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 927/1871] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:38.167844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a4c7b-42bb-442c-833b-697beadaf1ad service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:38.183382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421e8e7-940b-4181-b4d7-1ae3ca6c11d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:38.183382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421e8e7-940b-4181-b4d7-1ae3ca6c11d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:38.183382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421e8e7-940b-4181-b4d7-1ae3ca6c11d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:38.183382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421e8e7-940b-4181-b4d7-1ae3ca6c11d4 admin admin] PATH_INFO: `/v3/projects/1a10385d8c3743c3bc48cc3b39b53e0d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:38.183680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421e8e7-940b-4181-b4d7-1ae3ca6c11d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a10385d8c3743c3bc48cc3b39b53e0d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:38.183680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421e8e7-940b-4181-b4d7-1ae3ca6c11d4 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:38.310920 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f421e8e7-940b-4181-b4d7-1ae3ca6c11d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a10385d8c3743c3bc48cc3b39b53e0d', 'request_id': 'req-f421e8e7-940b-4181-b4d7-1ae3ca6c11d4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:38.380556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8ca60-2478-45bc-a2b7-39b045cad2d3 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:38.382352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-51f8ca60-2478-45bc-a2b7-39b045cad2d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:38.393348 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8ca60-2478-45bc-a2b7-39b045cad2d3 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:38.444176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776549f0-d570-4f14-bad3-506d791c94f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:38.444789 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776549f0-d570-4f14-bad3-506d791c94f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:38.445072 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776549f0-d570-4f14-bad3-506d791c94f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:38.445347 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776549f0-d570-4f14-bad3-506d791c94f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:38.447093 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-776549f0-d570-4f14-bad3-506d791c94f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:38.519302 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776549f0-d570-4f14-bad3-506d791c94f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:38.521226 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f421e8e7-940b-4181-b4d7-1ae3ca6c11d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a10385d8c3743c3bc48cc3b39b53e0d', 'request_id': 'req-f421e8e7-940b-4181-b4d7-1ae3ca6c11d4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:38.531411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776549f0-d570-4f14-bad3-506d791c94f9 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:38.533275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f421e8e7-940b-4181-b4d7-1ae3ca6c11d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a10385d8c3743c3bc48cc3b39b53e0d', 'request_id': 'req-f421e8e7-940b-4181-b4d7-1ae3ca6c11d4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:38.551056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 928/1872] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:25:38.554671 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f421e8e7-940b-4181-b4d7-1ae3ca6c11d4 admin admin] The token cache is being invalidate because project 1a10385d8c3743c3bc48cc3b39b53e0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:38.555039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f421e8e7-940b-4181-b4d7-1ae3ca6c11d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:38.555814 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f421e8e7-940b-4181-b4d7-1ae3ca6c11d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:38.557883 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 945/1873] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:38 2022] DELETE /identity/v3/projects/1a10385d8c3743c3bc48cc3b39b53e0d => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:25:38.565125 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e79d7a-13cc-4f12-be70-c93d1eb52d42 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:38.565808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e79d7a-13cc-4f12-be70-c93d1eb52d42 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:38.566092 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e79d7a-13cc-4f12-be70-c93d1eb52d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:38.590445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-63e79d7a-13cc-4f12-be70-c93d1eb52d42 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:38.592253 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-63e79d7a-13cc-4f12-be70-c93d1eb52d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:38.664663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 929/1874] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:38.672464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2eded-9011-4324-980c-cf06a923972c None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:38.672827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2eded-9011-4324-980c-cf06a923972c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:38.673115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2eded-9011-4324-980c-cf06a923972c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:38.693645 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-cbd2eded-9011-4324-980c-cf06a923972c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:38.694944 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-cbd2eded-9011-4324-980c-cf06a923972c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:38.704042 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 946/1875] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:38.707991 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e79d7a-13cc-4f12-be70-c93d1eb52d42 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:38.718295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e79d7a-13cc-4f12-be70-c93d1eb52d42 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:38.723025 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fb8a9-08f0-4d30-80d5-7872613c5e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:38.723678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fb8a9-08f0-4d30-80d5-7872613c5e6f admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:38.723968 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fb8a9-08f0-4d30-80d5-7872613c5e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:38.724227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fb8a9-08f0-4d30-80d5-7872613c5e6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:38.724933 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fb8a9-08f0-4d30-80d5-7872613c5e6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:38.725274 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fb8a9-08f0-4d30-80d5-7872613c5e6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:38.726866 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fb8a9-08f0-4d30-80d5-7872613c5e6f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:38.745887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 930/1876] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:25:38.746940 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2eded-9011-4324-980c-cf06a923972c None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:38.756532 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2eded-9011-4324-980c-cf06a923972c None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:38.761768 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb0f69-4b68-4710-bf25-05c8f8f90e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:38.762388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb0f69-4b68-4710-bf25-05c8f8f90e51 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:38.762671 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb0f69-4b68-4710-bf25-05c8f8f90e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:38.762971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb0f69-4b68-4710-bf25-05c8f8f90e51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:38.763656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb0f69-4b68-4710-bf25-05c8f8f90e51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:38.765726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb0f69-4b68-4710-bf25-05c8f8f90e51 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:38.948750 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 947/1877] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:38 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:38.963728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fb8a9-08f0-4d30-80d5-7872613c5e6f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:38.976684 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fb8a9-08f0-4d30-80d5-7872613c5e6f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:39.038911 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d8448b-18f3-419d-b3c0-6f708075a836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:39.039764 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8448b-18f3-419d-b3c0-6f708075a836 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:39.039764 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8448b-18f3-419d-b3c0-6f708075a836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:39.040091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8448b-18f3-419d-b3c0-6f708075a836 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:39.041020 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d8448b-18f3-419d-b3c0-6f708075a836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:39.042953 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d8448b-18f3-419d-b3c0-6f708075a836 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:39.164915 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 931/1878] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:38 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:39.170393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb0f69-4b68-4710-bf25-05c8f8f90e51 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:39.182729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb0f69-4b68-4710-bf25-05c8f8f90e51 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:39.202460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6270e89-4e6f-4310-a144-21fb85f10637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:39.202460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6270e89-4e6f-4310-a144-21fb85f10637 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:39.202460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6270e89-4e6f-4310-a144-21fb85f10637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:39.202460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6270e89-4e6f-4310-a144-21fb85f10637 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:39.204392 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6270e89-4e6f-4310-a144-21fb85f10637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:39.206547 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6270e89-4e6f-4310-a144-21fb85f10637 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:39.220012 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 948/1879] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:39.224294 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d8448b-18f3-419d-b3c0-6f708075a836 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:39.238492 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d8448b-18f3-419d-b3c0-6f708075a836 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:39.243531 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8fe0e-3c64-48ad-a2f5-f9ef4b18238d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:39.244682 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8fe0e-3c64-48ad-a2f5-f9ef4b18238d admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:39.244958 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8fe0e-3c64-48ad-a2f5-f9ef4b18238d admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:39.245236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8fe0e-3c64-48ad-a2f5-f9ef4b18238d admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:39.245917 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8fe0e-3c64-48ad-a2f5-f9ef4b18238d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:39.247615 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8fe0e-3c64-48ad-a2f5-f9ef4b18238d admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:39.261161 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 932/1880] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:25:39.265453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6270e89-4e6f-4310-a144-21fb85f10637 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:39.276194 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6270e89-4e6f-4310-a144-21fb85f10637 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:39.282176 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70235bdb-f75e-466e-b05c-a87e01136679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:39.282570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70235bdb-f75e-466e-b05c-a87e01136679 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:39.282903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70235bdb-f75e-466e-b05c-a87e01136679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:39.283381 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70235bdb-f75e-466e-b05c-a87e01136679 admin admin] PATH_INFO: `/v3/projects/5f024fc9947141538089c63b2e8ef6a0/users/8e3a1fac73e3474fbd7ce268554726aa/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:39.300499 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70235bdb-f75e-466e-b05c-a87e01136679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f024fc9947141538089c63b2e8ef6a0, user_id=8e3a1fac73e3474fbd7ce268554726aa, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:39.303224 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70235bdb-f75e-466e-b05c-a87e01136679 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:39.391593 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 949/1881] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:39 2022] PUT /identity/v3/projects/5f024fc9947141538089c63b2e8ef6a0/users/8e3a1fac73e3474fbd7ce268554726aa/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:25:39.402141 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abe985-e08f-4fa8-9c0e-a4b1baf9b117 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:39.402456 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abe985-e08f-4fa8-9c0e-a4b1baf9b117 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:39.402741 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abe985-e08f-4fa8-9c0e-a4b1baf9b117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:39.433238 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-f2abe985-e08f-4fa8-9c0e-a4b1baf9b117 None None] MFA Rules not processed for user `8e3a1fac73e3474fbd7ce268554726aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:39.434466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f2abe985-e08f-4fa8-9c0e-a4b1baf9b117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:39.508209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 933/1882] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:39.520296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70235bdb-f75e-466e-b05c-a87e01136679 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:39.606642 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70235bdb-f75e-466e-b05c-a87e01136679 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:39.609462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947f512-a46a-4515-8e6c-ead34f1ed6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:39.610395 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947f512-a46a-4515-8e6c-ead34f1ed6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:39.611194 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947f512-a46a-4515-8e6c-ead34f1ed6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:39.611483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947f512-a46a-4515-8e6c-ead34f1ed6f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:39.612181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947f512-a46a-4515-8e6c-ead34f1ed6f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:39.614416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947f512-a46a-4515-8e6c-ead34f1ed6f3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:39.721396 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 950/1883] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:39 2022] POST /identity/v3/projects => generated 387 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:39.724951 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2abe985-e08f-4fa8-9c0e-a4b1baf9b117 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:39.736660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2abe985-e08f-4fa8-9c0e-a4b1baf9b117 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:39.798169 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e0733-a176-4f62-80d2-512f5221dd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:39.798780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e0733-a176-4f62-80d2-512f5221dd5b admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:39.799095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e0733-a176-4f62-80d2-512f5221dd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:39.799405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e0733-a176-4f62-80d2-512f5221dd5b admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:39.800061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e0733-a176-4f62-80d2-512f5221dd5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:39.801955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e0733-a176-4f62-80d2-512f5221dd5b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:39.875783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=83,keystone:DELETE=12,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:39.886919 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947f512-a46a-4515-8e6c-ead34f1ed6f3 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:39.887474 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 934/1884] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:39 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:39.905543 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947f512-a46a-4515-8e6c-ead34f1ed6f3 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:39.911298 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce266ea9-2a0e-4a80-bfa1-bc380d74d1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:39.913183 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce266ea9-2a0e-4a80-bfa1-bc380d74d1f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:39.913886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce266ea9-2a0e-4a80-bfa1-bc380d74d1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:39.915538 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce266ea9-2a0e-4a80-bfa1-bc380d74d1f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:39.916932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce266ea9-2a0e-4a80-bfa1-bc380d74d1f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:39.921049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce266ea9-2a0e-4a80-bfa1-bc380d74d1f7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:39.946907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 951/1885] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:25:39.953707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e0733-a176-4f62-80d2-512f5221dd5b admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:39.959386 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e0733-a176-4f62-80d2-512f5221dd5b admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:39.964615 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19711b-5ef1-4c89-b9c6-af5b76375223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:39.965290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19711b-5ef1-4c89-b9c6-af5b76375223 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:39.965573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19711b-5ef1-4c89-b9c6-af5b76375223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:39.966116 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19711b-5ef1-4c89-b9c6-af5b76375223 admin admin] PATH_INFO: `/v3/projects/6b0dc573bb354e9c867dbb630d815b7b/users/1dca4507e0104d6684931ad1a23b1f2c/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:39.982334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19711b-5ef1-4c89-b9c6-af5b76375223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b0dc573bb354e9c867dbb630d815b7b, user_id=1dca4507e0104d6684931ad1a23b1f2c, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:39.985254 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19711b-5ef1-4c89-b9c6-af5b76375223 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:40.155866 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 935/1886] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:39 2022] PUT /identity/v3/projects/6b0dc573bb354e9c867dbb630d815b7b/users/1dca4507e0104d6684931ad1a23b1f2c/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:40.159835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce266ea9-2a0e-4a80-bfa1-bc380d74d1f7 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:40.172109 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce266ea9-2a0e-4a80-bfa1-bc380d74d1f7 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:40.256448 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a808d-7d6a-4c68-8e4f-b9856ec50aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:40.257205 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a808d-7d6a-4c68-8e4f-b9856ec50aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:40.257549 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a808d-7d6a-4c68-8e4f-b9856ec50aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:40.257817 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a808d-7d6a-4c68-8e4f-b9856ec50aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:40.258477 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a808d-7d6a-4c68-8e4f-b9856ec50aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:40.264113 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a808d-7d6a-4c68-8e4f-b9856ec50aa8 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:40.264519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:40.278483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 952/1887] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:40.279491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19711b-5ef1-4c89-b9c6-af5b76375223 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:40.289913 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19711b-5ef1-4c89-b9c6-af5b76375223 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:40.295396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccde1d-a7e2-4a4f-8d30-6f505cff69c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:40.296069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccde1d-a7e2-4a4f-8d30-6f505cff69c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:40.296348 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccde1d-a7e2-4a4f-8d30-6f505cff69c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:40.296622 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccde1d-a7e2-4a4f-8d30-6f505cff69c9 admin admin] PATH_INFO: `/v3/projects/6b0dc573bb354e9c867dbb630d815b7b/users/1dca4507e0104d6684931ad1a23b1f2c/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:40.298996 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccde1d-a7e2-4a4f-8d30-6f505cff69c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b0dc573bb354e9c867dbb630d815b7b, user_id=1dca4507e0104d6684931ad1a23b1f2c, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:40.301986 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccde1d-a7e2-4a4f-8d30-6f505cff69c9 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:40.314379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 936/1888] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:40 2022] PUT /identity/v3/projects/6b0dc573bb354e9c867dbb630d815b7b/users/1dca4507e0104d6684931ad1a23b1f2c/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:40.325934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007b854-9496-469c-8e55-eea40714befb None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:40.326225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007b854-9496-469c-8e55-eea40714befb None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:40.327423 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007b854-9496-469c-8e55-eea40714befb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:40.357522 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-6007b854-9496-469c-8e55-eea40714befb None None] MFA Rules not processed for user `1dca4507e0104d6684931ad1a23b1f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:40.358639 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6007b854-9496-469c-8e55-eea40714befb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:40.443584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 953/1889] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:40.454774 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccde1d-a7e2-4a4f-8d30-6f505cff69c9 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:40.456671 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-bdccde1d-a7e2-4a4f-8d30-6f505cff69c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:40.513566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccde1d-a7e2-4a4f-8d30-6f505cff69c9 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:40.517271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98268d3b-fb5f-4f35-aecb-7139d43e7c69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:40.517995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98268d3b-fb5f-4f35-aecb-7139d43e7c69 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:40.518168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98268d3b-fb5f-4f35-aecb-7139d43e7c69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:40.518443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98268d3b-fb5f-4f35-aecb-7139d43e7c69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:40.528169 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98268d3b-fb5f-4f35-aecb-7139d43e7c69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:40.529592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98268d3b-fb5f-4f35-aecb-7139d43e7c69 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:40.538968 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 937/1890] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:40 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:40.567636 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007b854-9496-469c-8e55-eea40714befb None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:40.632529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007b854-9496-469c-8e55-eea40714befb None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:40.637255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e989f-c906-4d37-ac8a-c80c1807dd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:40.637859 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e989f-c906-4d37-ac8a-c80c1807dd2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:40.638134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e989f-c906-4d37-ac8a-c80c1807dd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:40.638414 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e989f-c906-4d37-ac8a-c80c1807dd2a admin admin] PATH_INFO: `/v3/users/8e3a1fac73e3474fbd7ce268554726aa` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:40.639801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e989f-c906-4d37-ac8a-c80c1807dd2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e3a1fac73e3474fbd7ce268554726aa)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:40.641735 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e989f-c906-4d37-ac8a-c80c1807dd2a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:40.669757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:40.913275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-296e989f-c906-4d37-ac8a-c80c1807dd2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e3a1fac73e3474fbd7ce268554726aa'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:40.925921 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-296e989f-c906-4d37-ac8a-c80c1807dd2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e3a1fac73e3474fbd7ce268554726aa'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:40.933532 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-296e989f-c906-4d37-ac8a-c80c1807dd2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e3a1fac73e3474fbd7ce268554726aa'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:41.082157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-296e989f-c906-4d37-ac8a-c80c1807dd2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e3a1fac73e3474fbd7ce268554726aa'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:41.085781 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-296e989f-c906-4d37-ac8a-c80c1807dd2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e3a1fac73e3474fbd7ce268554726aa'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:41.099901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 954/1891] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:40 2022] DELETE /identity/v3/users/8e3a1fac73e3474fbd7ce268554726aa => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:41.104221 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f78ad-b92b-415a-bd53-a966ab50f3b2 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:41.104510 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f78ad-b92b-415a-bd53-a966ab50f3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:41.104793 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f78ad-b92b-415a-bd53-a966ab50f3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:41.133862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-191f78ad-b92b-415a-bd53-a966ab50f3b2 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:41.135661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-191f78ad-b92b-415a-bd53-a966ab50f3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:41.196825 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 938/1892] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:41.207342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e989f-c906-4d37-ac8a-c80c1807dd2a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:41.208863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-296e989f-c906-4d37-ac8a-c80c1807dd2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:41.294827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e989f-c906-4d37-ac8a-c80c1807dd2a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:41.307153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7e532-bd80-482f-87fb-e3747fd1a792 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:41.307153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7e532-bd80-482f-87fb-e3747fd1a792 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:41.307153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7e532-bd80-482f-87fb-e3747fd1a792 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:41.307153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7e532-bd80-482f-87fb-e3747fd1a792 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:41.311892 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7e532-bd80-482f-87fb-e3747fd1a792 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:41.313479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7e532-bd80-482f-87fb-e3747fd1a792 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:41.332429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 955/1893] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:41.868125 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f78ad-b92b-415a-bd53-a966ab50f3b2 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:41.871668 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-191f78ad-b92b-415a-bd53-a966ab50f3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:41.881223 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:41.891224 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f78ad-b92b-415a-bd53-a966ab50f3b2 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:41.896997 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2ff6b-1f33-41ff-8334-fc778745fa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:41.897702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2ff6b-1f33-41ff-8334-fc778745fa8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:41.898213 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2ff6b-1f33-41ff-8334-fc778745fa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:41.898562 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2ff6b-1f33-41ff-8334-fc778745fa8e admin admin] PATH_INFO: `/v3/projects/5f024fc9947141538089c63b2e8ef6a0` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:41.900072 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2ff6b-1f33-41ff-8334-fc778745fa8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f024fc9947141538089c63b2e8ef6a0)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:41.902931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2ff6b-1f33-41ff-8334-fc778745fa8e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:42.024383 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-22c2ff6b-1f33-41ff-8334-fc778745fa8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f024fc9947141538089c63b2e8ef6a0', 'request_id': 'req-22c2ff6b-1f33-41ff-8334-fc778745fa8e'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:42.032660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-22c2ff6b-1f33-41ff-8334-fc778745fa8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f024fc9947141538089c63b2e8ef6a0', 'request_id': 'req-22c2ff6b-1f33-41ff-8334-fc778745fa8e'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:42.178288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-22c2ff6b-1f33-41ff-8334-fc778745fa8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f024fc9947141538089c63b2e8ef6a0', 'request_id': 'req-22c2ff6b-1f33-41ff-8334-fc778745fa8e'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:42.190729 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-22c2ff6b-1f33-41ff-8334-fc778745fa8e admin admin] The token cache is being invalidate because project 5f024fc9947141538089c63b2e8ef6a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:42.191132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-22c2ff6b-1f33-41ff-8334-fc778745fa8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:42.191810 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-22c2ff6b-1f33-41ff-8334-fc778745fa8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:42.193329 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 939/1894] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:41 2022] DELETE /identity/v3/projects/5f024fc9947141538089c63b2e8ef6a0 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:42.197369 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7e532-bd80-482f-87fb-e3747fd1a792 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:42.207420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7e532-bd80-482f-87fb-e3747fd1a792 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:42.264084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b65cfc-8d14-40a8-95e9-9b3b72ccc968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:42.264727 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b65cfc-8d14-40a8-95e9-9b3b72ccc968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:42.265010 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b65cfc-8d14-40a8-95e9-9b3b72ccc968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:42.265286 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b65cfc-8d14-40a8-95e9-9b3b72ccc968 admin admin] PATH_INFO: `/v3/users/1dca4507e0104d6684931ad1a23b1f2c` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:42.266427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b65cfc-8d14-40a8-95e9-9b3b72ccc968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dca4507e0104d6684931ad1a23b1f2c)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:42.268436 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b65cfc-8d14-40a8-95e9-9b3b72ccc968 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:42.501387 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-97b65cfc-8d14-40a8-95e9-9b3b72ccc968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dca4507e0104d6684931ad1a23b1f2c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:42.514747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-97b65cfc-8d14-40a8-95e9-9b3b72ccc968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dca4507e0104d6684931ad1a23b1f2c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:42.522983 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-97b65cfc-8d14-40a8-95e9-9b3b72ccc968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dca4507e0104d6684931ad1a23b1f2c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:42.629799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-97b65cfc-8d14-40a8-95e9-9b3b72ccc968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dca4507e0104d6684931ad1a23b1f2c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:42.630438 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-97b65cfc-8d14-40a8-95e9-9b3b72ccc968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dca4507e0104d6684931ad1a23b1f2c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:42.637561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 956/1895] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:42 2022] DELETE /identity/v3/users/1dca4507e0104d6684931ad1a23b1f2c => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:43.024861 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:43.152732 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2ff6b-1f33-41ff-8334-fc778745fa8e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:43.154380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-22c2ff6b-1f33-41ff-8334-fc778745fa8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:43.171030 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2ff6b-1f33-41ff-8334-fc778745fa8e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:43.237007 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca489dc1-ef8b-4fa8-814b-c6a33f6c7a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:43.237568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489dc1-ef8b-4fa8-814b-c6a33f6c7a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:43.237848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489dc1-ef8b-4fa8-814b-c6a33f6c7a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:43.238122 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489dc1-ef8b-4fa8-814b-c6a33f6c7a9f admin admin] PATH_INFO: `/v3/projects/6b0dc573bb354e9c867dbb630d815b7b` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:43.239282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca489dc1-ef8b-4fa8-814b-c6a33f6c7a9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b0dc573bb354e9c867dbb630d815b7b)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:43.241305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca489dc1-ef8b-4fa8-814b-c6a33f6c7a9f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:43.406520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:43.488407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ca489dc1-ef8b-4fa8-814b-c6a33f6c7a9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b0dc573bb354e9c867dbb630d815b7b', 'request_id': 'req-ca489dc1-ef8b-4fa8-814b-c6a33f6c7a9f'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:43.495275 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ca489dc1-ef8b-4fa8-814b-c6a33f6c7a9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b0dc573bb354e9c867dbb630d815b7b', 'request_id': 'req-ca489dc1-ef8b-4fa8-814b-c6a33f6c7a9f'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:43.623705 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ca489dc1-ef8b-4fa8-814b-c6a33f6c7a9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b0dc573bb354e9c867dbb630d815b7b', 'request_id': 'req-ca489dc1-ef8b-4fa8-814b-c6a33f6c7a9f'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:43.635770 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ca489dc1-ef8b-4fa8-814b-c6a33f6c7a9f admin admin] The token cache is being invalidate because project 6b0dc573bb354e9c867dbb630d815b7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:43.636057 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ca489dc1-ef8b-4fa8-814b-c6a33f6c7a9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:43.636695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ca489dc1-ef8b-4fa8-814b-c6a33f6c7a9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:43.638178 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 940/1896] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:43 2022] DELETE /identity/v3/projects/6b0dc573bb354e9c867dbb630d815b7b => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:43.643144 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f8078-678d-4cf2-ae52-17f962faa1e7 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:43.643467 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f8078-678d-4cf2-ae52-17f962faa1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:43.643743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f8078-678d-4cf2-ae52-17f962faa1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:43.665815 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4d5f8078-678d-4cf2-ae52-17f962faa1e7 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:43.672516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4d5f8078-678d-4cf2-ae52-17f962faa1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:43.726649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 957/1897] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:43.735082 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50495b2-4bc2-4d0d-ace9-52864b7074ca None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:43.735398 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50495b2-4bc2-4d0d-ace9-52864b7074ca None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:43.735694 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50495b2-4bc2-4d0d-ace9-52864b7074ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:43.755308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-c50495b2-4bc2-4d0d-ace9-52864b7074ca None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:43.756506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c50495b2-4bc2-4d0d-ace9-52864b7074ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:43.764961 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 941/1898] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:43.769240 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f8078-678d-4cf2-ae52-17f962faa1e7 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:43.782122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f8078-678d-4cf2-ae52-17f962faa1e7 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:43.788230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6148e-0aa5-4e3a-a24e-723a8475c0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:43.788872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6148e-0aa5-4e3a-a24e-723a8475c0c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:43.789160 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6148e-0aa5-4e3a-a24e-723a8475c0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:43.789434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6148e-0aa5-4e3a-a24e-723a8475c0c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:43.790105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e6148e-0aa5-4e3a-a24e-723a8475c0c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:43.790416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e6148e-0aa5-4e3a-a24e-723a8475c0c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:43.792434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e6148e-0aa5-4e3a-a24e-723a8475c0c8 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:43.814471 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 958/1899] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:25:43.826466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50495b2-4bc2-4d0d-ace9-52864b7074ca None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:43.829867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50495b2-4bc2-4d0d-ace9-52864b7074ca None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:43.839698 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431dfdd-c432-42a3-b37b-00ca189e3b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:43.841295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431dfdd-c432-42a3-b37b-00ca189e3b2f admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:43.842067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431dfdd-c432-42a3-b37b-00ca189e3b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:43.842615 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431dfdd-c432-42a3-b37b-00ca189e3b2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:43.843783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431dfdd-c432-42a3-b37b-00ca189e3b2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:43.847825 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431dfdd-c432-42a3-b37b-00ca189e3b2f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:44.007812 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 942/1900] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:43 2022] POST /identity/v3/projects => generated 387 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:44.012730 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6148e-0aa5-4e3a-a24e-723a8475c0c8 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:44.022908 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6148e-0aa5-4e3a-a24e-723a8475c0c8 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:44.101565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00ad4a-8d8a-4419-8baa-f98e235477ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:44.102197 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00ad4a-8d8a-4419-8baa-f98e235477ea admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:44.102480 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00ad4a-8d8a-4419-8baa-f98e235477ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:44.102757 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00ad4a-8d8a-4419-8baa-f98e235477ea admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:44.103454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a00ad4a-8d8a-4419-8baa-f98e235477ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:44.105403 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a00ad4a-8d8a-4419-8baa-f98e235477ea admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:44.224000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 959/1901] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:44 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:25:44.228219 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431dfdd-c432-42a3-b37b-00ca189e3b2f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:44.248824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431dfdd-c432-42a3-b37b-00ca189e3b2f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:44.254105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b20121-972a-4420-8579-b7f39823b6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:44.254732 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b20121-972a-4420-8579-b7f39823b6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:44.255014 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b20121-972a-4420-8579-b7f39823b6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:44.255328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b20121-972a-4420-8579-b7f39823b6a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:44.255979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b20121-972a-4420-8579-b7f39823b6a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:44.257590 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b20121-972a-4420-8579-b7f39823b6a1 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:44.268576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 943/1902] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:25:44.272507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00ad4a-8d8a-4419-8baa-f98e235477ea admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:44.282506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00ad4a-8d8a-4419-8baa-f98e235477ea admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:44.287922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6f184-293d-4ef6-86fb-5b56dd85eeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:44.288563 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6f184-293d-4ef6-86fb-5b56dd85eeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:44.288861 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6f184-293d-4ef6-86fb-5b56dd85eeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:44.289122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6f184-293d-4ef6-86fb-5b56dd85eeb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:44.289795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6f184-293d-4ef6-86fb-5b56dd85eeb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:44.291408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6f184-293d-4ef6-86fb-5b56dd85eeb3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:44.302040 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 960/1903] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:44.306491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b20121-972a-4420-8579-b7f39823b6a1 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:44.317086 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b20121-972a-4420-8579-b7f39823b6a1 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:44.321648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b73980-b1f9-401d-a54b-38458f6e1562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:44.322297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b73980-b1f9-401d-a54b-38458f6e1562 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:44.322580 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b73980-b1f9-401d-a54b-38458f6e1562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:44.323022 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b73980-b1f9-401d-a54b-38458f6e1562 admin admin] PATH_INFO: `/v3/projects/6578bd0486ce401a9e7d07c25ffb018e/users/eb63baba4a9e485c873c0a0ece37193a/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:44.340086 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b73980-b1f9-401d-a54b-38458f6e1562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6578bd0486ce401a9e7d07c25ffb018e, user_id=eb63baba4a9e485c873c0a0ece37193a, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:44.343300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b73980-b1f9-401d-a54b-38458f6e1562 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:44.465868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 944/1904] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:44 2022] PUT /identity/v3/projects/6578bd0486ce401a9e7d07c25ffb018e/users/eb63baba4a9e485c873c0a0ece37193a/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:44.476730 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fb676-d2ea-4a36-be2a-aa7e12830267 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:44.477361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fb676-d2ea-4a36-be2a-aa7e12830267 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:44.477361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fb676-d2ea-4a36-be2a-aa7e12830267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:44.505750 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ae3fb676-d2ea-4a36-be2a-aa7e12830267 None None] MFA Rules not processed for user `eb63baba4a9e485c873c0a0ece37193a`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:44.506978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ae3fb676-d2ea-4a36-be2a-aa7e12830267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:44.569791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 961/1905] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:44.574249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b73980-b1f9-401d-a54b-38458f6e1562 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:44.635871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b73980-b1f9-401d-a54b-38458f6e1562 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:44.640484 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d055c-4aef-43c1-aec3-18ae5d10d900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:44.641164 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d055c-4aef-43c1-aec3-18ae5d10d900 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:44.641369 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d055c-4aef-43c1-aec3-18ae5d10d900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:44.641642 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d055c-4aef-43c1-aec3-18ae5d10d900 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:44.642264 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d055c-4aef-43c1-aec3-18ae5d10d900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:44.644189 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d055c-4aef-43c1-aec3-18ae5d10d900 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:44.739700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 945/1906] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:44 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:44.743791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fb676-d2ea-4a36-be2a-aa7e12830267 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:44.755215 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fb676-d2ea-4a36-be2a-aa7e12830267 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:44.810130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35185423-89ba-481b-91df-b789c97047d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:44.810778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35185423-89ba-481b-91df-b789c97047d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:44.811068 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35185423-89ba-481b-91df-b789c97047d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:44.811368 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35185423-89ba-481b-91df-b789c97047d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:44.812013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35185423-89ba-481b-91df-b789c97047d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:44.813922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35185423-89ba-481b-91df-b789c97047d9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:44.905874 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 962/1907] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:44 2022] POST /identity/v3/users => generated 300 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:44.910735 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d055c-4aef-43c1-aec3-18ae5d10d900 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:44.922768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d055c-4aef-43c1-aec3-18ae5d10d900 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:44.927511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e343e6-acee-4440-bf26-e34a9fb5cec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:44.928185 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e343e6-acee-4440-bf26-e34a9fb5cec8 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:44.928546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e343e6-acee-4440-bf26-e34a9fb5cec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:44.928883 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e343e6-acee-4440-bf26-e34a9fb5cec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:44.929586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e343e6-acee-4440-bf26-e34a9fb5cec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:44.931455 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e343e6-acee-4440-bf26-e34a9fb5cec8 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:44.942836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 946/1908] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:44.947060 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35185423-89ba-481b-91df-b789c97047d9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:44.957478 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35185423-89ba-481b-91df-b789c97047d9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:44.962018 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ed5ee-a00d-4149-a72d-dd11c9b195dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:44.962675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ed5ee-a00d-4149-a72d-dd11c9b195dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:44.962955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ed5ee-a00d-4149-a72d-dd11c9b195dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:44.963270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ed5ee-a00d-4149-a72d-dd11c9b195dd admin admin] PATH_INFO: `/v3/projects/23f81033814949a6b45725f815777afd/users/f8a4158ee0d84a6597809750ccf7e747/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:44.978059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ed5ee-a00d-4149-a72d-dd11c9b195dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=23f81033814949a6b45725f815777afd, user_id=f8a4158ee0d84a6597809750ccf7e747, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:44.980761 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ed5ee-a00d-4149-a72d-dd11c9b195dd admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:45.046696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 963/1909] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:44 2022] PUT /identity/v3/projects/23f81033814949a6b45725f815777afd/users/f8a4158ee0d84a6597809750ccf7e747/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:45.050719 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e343e6-acee-4440-bf26-e34a9fb5cec8 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:45.061489 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e343e6-acee-4440-bf26-e34a9fb5cec8 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:45.130749 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c88f5e-d0b4-4556-8a82-d60a78e24872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:45.131361 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c88f5e-d0b4-4556-8a82-d60a78e24872 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:45.131642 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c88f5e-d0b4-4556-8a82-d60a78e24872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:45.131963 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c88f5e-d0b4-4556-8a82-d60a78e24872 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:45.132666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c88f5e-d0b4-4556-8a82-d60a78e24872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:45.134356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c88f5e-d0b4-4556-8a82-d60a78e24872 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:45.150092 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 947/1910] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:45.151487 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ed5ee-a00d-4149-a72d-dd11c9b195dd admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:45.162331 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ed5ee-a00d-4149-a72d-dd11c9b195dd admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:45.167433 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:45.167600 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93290f8-74ce-4e00-a824-79926d146e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:45.168286 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93290f8-74ce-4e00-a824-79926d146e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:45.168596 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93290f8-74ce-4e00-a824-79926d146e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:45.168871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93290f8-74ce-4e00-a824-79926d146e93 admin admin] PATH_INFO: `/v3/projects/23f81033814949a6b45725f815777afd/users/f8a4158ee0d84a6597809750ccf7e747/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:45.170785 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93290f8-74ce-4e00-a824-79926d146e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23f81033814949a6b45725f815777afd, user_id=f8a4158ee0d84a6597809750ccf7e747, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:45.173449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93290f8-74ce-4e00-a824-79926d146e93 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:45.185432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 964/1911] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:45 2022] PUT /identity/v3/projects/23f81033814949a6b45725f815777afd/users/f8a4158ee0d84a6597809750ccf7e747/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:25:45.197081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd9712-8b0a-4709-963b-dacae386f61b None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:45.197602 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd9712-8b0a-4709-963b-dacae386f61b None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:45.198493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd9712-8b0a-4709-963b-dacae386f61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:45.228029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-84bd9712-8b0a-4709-963b-dacae386f61b None None] MFA Rules not processed for user `f8a4158ee0d84a6597809750ccf7e747`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:45.229804 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-84bd9712-8b0a-4709-963b-dacae386f61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:45.306234 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 948/1912] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:45.318408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93290f8-74ce-4e00-a824-79926d146e93 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:45.320765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e93290f8-74ce-4e00-a824-79926d146e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:45.376046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:45.394707 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93290f8-74ce-4e00-a824-79926d146e93 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:45.399781 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518ca2f-cf4a-4639-bb47-34ce1efada07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:45.404629 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518ca2f-cf4a-4639-bb47-34ce1efada07 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:45.404629 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518ca2f-cf4a-4639-bb47-34ce1efada07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:45.404629 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518ca2f-cf4a-4639-bb47-34ce1efada07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:45.410909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518ca2f-cf4a-4639-bb47-34ce1efada07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:45.412370 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518ca2f-cf4a-4639-bb47-34ce1efada07 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:45.423284 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 965/1913] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:45 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:45.507787 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd9712-8b0a-4709-963b-dacae386f61b None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:45.574841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd9712-8b0a-4709-963b-dacae386f61b None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:45.579874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5cf1b-d3ef-45dd-a4b6-e8f2d28fc242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:45.580535 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5cf1b-d3ef-45dd-a4b6-e8f2d28fc242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:45.580808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5cf1b-d3ef-45dd-a4b6-e8f2d28fc242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:45.581079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5cf1b-d3ef-45dd-a4b6-e8f2d28fc242 admin admin] PATH_INFO: `/v3/users/eb63baba4a9e485c873c0a0ece37193a` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:45.582224 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5cf1b-d3ef-45dd-a4b6-e8f2d28fc242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb63baba4a9e485c873c0a0ece37193a)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:45.584372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5cf1b-d3ef-45dd-a4b6-e8f2d28fc242 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:45.797621 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-30b5cf1b-d3ef-45dd-a4b6-e8f2d28fc242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb63baba4a9e485c873c0a0ece37193a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:45.809426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-30b5cf1b-d3ef-45dd-a4b6-e8f2d28fc242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb63baba4a9e485c873c0a0ece37193a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:45.816192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-30b5cf1b-d3ef-45dd-a4b6-e8f2d28fc242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb63baba4a9e485c873c0a0ece37193a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:45.886533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-30b5cf1b-d3ef-45dd-a4b6-e8f2d28fc242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb63baba4a9e485c873c0a0ece37193a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:45.887209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-30b5cf1b-d3ef-45dd-a4b6-e8f2d28fc242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb63baba4a9e485c873c0a0ece37193a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:45.898224 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 949/1914] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:45 2022] DELETE /identity/v3/users/eb63baba4a9e485c873c0a0ece37193a => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:45.904297 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc5d66-15b3-41b8-8475-ee1eceade232 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:45.904789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc5d66-15b3-41b8-8475-ee1eceade232 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:45.904876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc5d66-15b3-41b8-8475-ee1eceade232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:45.929910 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-2bbc5d66-15b3-41b8-8475-ee1eceade232 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:45.931449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2bbc5d66-15b3-41b8-8475-ee1eceade232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:46.000546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 966/1915] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:46.011193 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5cf1b-d3ef-45dd-a4b6-e8f2d28fc242 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:46.012689 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-30b5cf1b-d3ef-45dd-a4b6-e8f2d28fc242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:46.076313 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5cf1b-d3ef-45dd-a4b6-e8f2d28fc242 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:46.080609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b7b69-f9d8-4b1e-8863-e69a3691603f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:46.081336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b7b69-f9d8-4b1e-8863-e69a3691603f service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:46.081725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b7b69-f9d8-4b1e-8863-e69a3691603f service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:46.082094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b7b69-f9d8-4b1e-8863-e69a3691603f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:46.093052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b7b69-f9d8-4b1e-8863-e69a3691603f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:46.095404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b7b69-f9d8-4b1e-8863-e69a3691603f service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:46.105371 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 950/1916] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:46.172890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:46.561362 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:46.610529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc5d66-15b3-41b8-8475-ee1eceade232 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:46.618416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2bbc5d66-15b3-41b8-8475-ee1eceade232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:46.632998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc5d66-15b3-41b8-8475-ee1eceade232 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:46.639045 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f212a-515e-4e45-bae6-4f18f0ac4745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:46.639822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f212a-515e-4e45-bae6-4f18f0ac4745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:46.640236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f212a-515e-4e45-bae6-4f18f0ac4745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:46.640589 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f212a-515e-4e45-bae6-4f18f0ac4745 admin admin] PATH_INFO: `/v3/projects/6578bd0486ce401a9e7d07c25ffb018e` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:46.641896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f212a-515e-4e45-bae6-4f18f0ac4745 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6578bd0486ce401a9e7d07c25ffb018e)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:46.644117 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f212a-515e-4e45-bae6-4f18f0ac4745 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:46.769516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-943f212a-515e-4e45-bae6-4f18f0ac4745 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6578bd0486ce401a9e7d07c25ffb018e', 'request_id': 'req-943f212a-515e-4e45-bae6-4f18f0ac4745'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:46.829234 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-943f212a-515e-4e45-bae6-4f18f0ac4745 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6578bd0486ce401a9e7d07c25ffb018e', 'request_id': 'req-943f212a-515e-4e45-bae6-4f18f0ac4745'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:46.836294 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-943f212a-515e-4e45-bae6-4f18f0ac4745 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6578bd0486ce401a9e7d07c25ffb018e', 'request_id': 'req-943f212a-515e-4e45-bae6-4f18f0ac4745'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:46.847045 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-943f212a-515e-4e45-bae6-4f18f0ac4745 admin admin] The token cache is being invalidate because project 6578bd0486ce401a9e7d07c25ffb018e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:46.847375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-943f212a-515e-4e45-bae6-4f18f0ac4745 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:46.848048 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-943f212a-515e-4e45-bae6-4f18f0ac4745 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:46.849565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 967/1917] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:46 2022] DELETE /identity/v3/projects/6578bd0486ce401a9e7d07c25ffb018e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:46.855887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b7b69-f9d8-4b1e-8863-e69a3691603f service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:46.866350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b7b69-f9d8-4b1e-8863-e69a3691603f service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:46.923778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045c648-9fb8-4c8e-abd8-62d8d2c78d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:46.924447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045c648-9fb8-4c8e-abd8-62d8d2c78d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:46.924747 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045c648-9fb8-4c8e-abd8-62d8d2c78d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:46.925027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045c648-9fb8-4c8e-abd8-62d8d2c78d6f admin admin] PATH_INFO: `/v3/users/f8a4158ee0d84a6597809750ccf7e747` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:46.926201 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045c648-9fb8-4c8e-abd8-62d8d2c78d6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8a4158ee0d84a6597809750ccf7e747)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:46.928171 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045c648-9fb8-4c8e-abd8-62d8d2c78d6f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:47.120848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a045c648-9fb8-4c8e-abd8-62d8d2c78d6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8a4158ee0d84a6597809750ccf7e747'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:47.133893 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a045c648-9fb8-4c8e-abd8-62d8d2c78d6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8a4158ee0d84a6597809750ccf7e747'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:47.141546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a045c648-9fb8-4c8e-abd8-62d8d2c78d6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8a4158ee0d84a6597809750ccf7e747'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:47.257908 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a045c648-9fb8-4c8e-abd8-62d8d2c78d6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8a4158ee0d84a6597809750ccf7e747'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:47.258553 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a045c648-9fb8-4c8e-abd8-62d8d2c78d6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8a4158ee0d84a6597809750ccf7e747'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:47.265015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 951/1918] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:46 2022] DELETE /identity/v3/users/f8a4158ee0d84a6597809750ccf7e747 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:47.675232 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:47.717528 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f212a-515e-4e45-bae6-4f18f0ac4745 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:47.719079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-943f212a-515e-4e45-bae6-4f18f0ac4745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:47.733484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f212a-515e-4e45-bae6-4f18f0ac4745 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:47.795049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:47.799806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108e0ac-14a7-48a3-8a37-819ddeb06dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:47.800680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108e0ac-14a7-48a3-8a37-819ddeb06dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:47.801027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108e0ac-14a7-48a3-8a37-819ddeb06dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:47.801362 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108e0ac-14a7-48a3-8a37-819ddeb06dd5 admin admin] PATH_INFO: `/v3/projects/23f81033814949a6b45725f815777afd` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:47.802651 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108e0ac-14a7-48a3-8a37-819ddeb06dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23f81033814949a6b45725f815777afd)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:47.805105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108e0ac-14a7-48a3-8a37-819ddeb06dd5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:47.915513 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8108e0ac-14a7-48a3-8a37-819ddeb06dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23f81033814949a6b45725f815777afd', 'request_id': 'req-8108e0ac-14a7-48a3-8a37-819ddeb06dd5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:47.983603 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8108e0ac-14a7-48a3-8a37-819ddeb06dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23f81033814949a6b45725f815777afd', 'request_id': 'req-8108e0ac-14a7-48a3-8a37-819ddeb06dd5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:47.990882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8108e0ac-14a7-48a3-8a37-819ddeb06dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23f81033814949a6b45725f815777afd', 'request_id': 'req-8108e0ac-14a7-48a3-8a37-819ddeb06dd5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:48.003277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8108e0ac-14a7-48a3-8a37-819ddeb06dd5 admin admin] The token cache is being invalidate because project 23f81033814949a6b45725f815777afd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:48.005685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8108e0ac-14a7-48a3-8a37-819ddeb06dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:48.005685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8108e0ac-14a7-48a3-8a37-819ddeb06dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:48.005846 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 968/1919] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:47 2022] DELETE /identity/v3/projects/23f81033814949a6b45725f815777afd => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:48.011428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bfe48f-4cb4-4d84-b590-c3fde8886dca None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:48.011905 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bfe48f-4cb4-4d84-b590-c3fde8886dca None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:48.011999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bfe48f-4cb4-4d84-b590-c3fde8886dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:48.032159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-88bfe48f-4cb4-4d84-b590-c3fde8886dca None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:48.033379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-88bfe48f-4cb4-4d84-b590-c3fde8886dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:48.099005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 952/1920] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:48.111197 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90ae1f-8fb4-475e-b66f-edc6e1f1b1c9 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:48.111197 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90ae1f-8fb4-475e-b66f-edc6e1f1b1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:48.111197 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90ae1f-8fb4-475e-b66f-edc6e1f1b1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:48.129921 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ca90ae1f-8fb4-475e-b66f-edc6e1f1b1c9 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:48.131149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ca90ae1f-8fb4-475e-b66f-edc6e1f1b1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:48.139715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 969/1921] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:48.144145 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bfe48f-4cb4-4d84-b590-c3fde8886dca None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:48.153786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bfe48f-4cb4-4d84-b590-c3fde8886dca None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:48.158787 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b73736-1923-439e-b0e1-a9312c71da0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:48.159447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b73736-1923-439e-b0e1-a9312c71da0c admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:48.159729 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b73736-1923-439e-b0e1-a9312c71da0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:48.160005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b73736-1923-439e-b0e1-a9312c71da0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:48.160662 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b73736-1923-439e-b0e1-a9312c71da0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:48.160968 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b73736-1923-439e-b0e1-a9312c71da0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:48.162577 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b73736-1923-439e-b0e1-a9312c71da0c admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:48.186387 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 953/1922] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:25:48.186553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90ae1f-8fb4-475e-b66f-edc6e1f1b1c9 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:48.188676 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90ae1f-8fb4-475e-b66f-edc6e1f1b1c9 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:48.193314 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d46e2-f4ff-455b-a307-82abc3b8525e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:48.193913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d46e2-f4ff-455b-a307-82abc3b8525e admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:48.194203 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d46e2-f4ff-455b-a307-82abc3b8525e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:48.194475 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d46e2-f4ff-455b-a307-82abc3b8525e admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:48.195112 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d46e2-f4ff-455b-a307-82abc3b8525e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:48.197028 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d46e2-f4ff-455b-a307-82abc3b8525e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:48.310763 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 970/1923] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:48 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:25:48.315206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b73736-1923-439e-b0e1-a9312c71da0c admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:48.326167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b73736-1923-439e-b0e1-a9312c71da0c admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:48.326853 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:48.383134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb24f1-9be7-49fd-84c5-9daf39aad4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:48.384376 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb24f1-9be7-49fd-84c5-9daf39aad4e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:48.384376 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb24f1-9be7-49fd-84c5-9daf39aad4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:48.384376 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb24f1-9be7-49fd-84c5-9daf39aad4e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:48.384985 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb24f1-9be7-49fd-84c5-9daf39aad4e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:48.387456 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb24f1-9be7-49fd-84c5-9daf39aad4e2 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:48.521041 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 954/1924] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:48 2022] POST /identity/v3/users => generated 284 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:48.533319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d46e2-f4ff-455b-a307-82abc3b8525e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:48.543543 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d46e2-f4ff-455b-a307-82abc3b8525e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:48.548149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d12eb9-f265-4f05-a7b1-ee6f49320e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:48.548759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d12eb9-f265-4f05-a7b1-ee6f49320e51 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:48.549085 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d12eb9-f265-4f05-a7b1-ee6f49320e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:48.549361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d12eb9-f265-4f05-a7b1-ee6f49320e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:48.550008 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d12eb9-f265-4f05-a7b1-ee6f49320e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:48.551633 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d12eb9-f265-4f05-a7b1-ee6f49320e51 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:48.561488 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 971/1925] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:25:48.565626 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb24f1-9be7-49fd-84c5-9daf39aad4e2 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:48.576274 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb24f1-9be7-49fd-84c5-9daf39aad4e2 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:48.580897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f3071-a3d8-47fb-9da1-dff6618a7646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:48.581513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f3071-a3d8-47fb-9da1-dff6618a7646 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:48.581772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f3071-a3d8-47fb-9da1-dff6618a7646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:48.582107 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f3071-a3d8-47fb-9da1-dff6618a7646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:48.582735 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f3071-a3d8-47fb-9da1-dff6618a7646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:48.584350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f3071-a3d8-47fb-9da1-dff6618a7646 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:48.595761 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 955/1926] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:48.599954 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d12eb9-f265-4f05-a7b1-ee6f49320e51 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:48.609822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d12eb9-f265-4f05-a7b1-ee6f49320e51 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:48.614339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9b975-34f0-4136-ac1c-5f1bd308bb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:48.614990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9b975-34f0-4136-ac1c-5f1bd308bb19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:48.615305 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9b975-34f0-4136-ac1c-5f1bd308bb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:48.615585 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9b975-34f0-4136-ac1c-5f1bd308bb19 admin admin] PATH_INFO: `/v3/projects/1f71d072c0244f5785d2830d4f4a7ff2/users/facd975a155a4d3a975f3989b6c72f6d/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:48.630773 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9b975-34f0-4136-ac1c-5f1bd308bb19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f71d072c0244f5785d2830d4f4a7ff2, user_id=facd975a155a4d3a975f3989b6c72f6d, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:48.633435 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9b975-34f0-4136-ac1c-5f1bd308bb19 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:48.738025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 972/1927] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:48 2022] PUT /identity/v3/projects/1f71d072c0244f5785d2830d4f4a7ff2/users/facd975a155a4d3a975f3989b6c72f6d/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:48.748962 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a95b4a-c1fe-4371-b966-1c148aec20b7 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:48.749249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a95b4a-c1fe-4371-b966-1c148aec20b7 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:48.749522 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a95b4a-c1fe-4371-b966-1c148aec20b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:48.779367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-f1a95b4a-c1fe-4371-b966-1c148aec20b7 None None] MFA Rules not processed for user `facd975a155a4d3a975f3989b6c72f6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:48.780617 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f1a95b4a-c1fe-4371-b966-1c148aec20b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:48.850128 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 956/1928] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:48.854763 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9b975-34f0-4136-ac1c-5f1bd308bb19 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:48.887249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:48.955772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9b975-34f0-4136-ac1c-5f1bd308bb19 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:48.960686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b29c3-f2ff-4576-9464-41d0da4afab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:48.961315 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b29c3-f2ff-4576-9464-41d0da4afab7 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:48.961597 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b29c3-f2ff-4576-9464-41d0da4afab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:48.961872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b29c3-f2ff-4576-9464-41d0da4afab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:48.962656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b29c3-f2ff-4576-9464-41d0da4afab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:48.964837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b29c3-f2ff-4576-9464-41d0da4afab7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:49.075909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 973/1929] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:48 2022] POST /identity/v3/projects => generated 367 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:49.080664 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a95b4a-c1fe-4371-b966-1c148aec20b7 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:49.093085 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a95b4a-c1fe-4371-b966-1c148aec20b7 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:49.170279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d50e99-0c0c-4c2d-8480-2c8a8b286dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:49.171346 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d50e99-0c0c-4c2d-8480-2c8a8b286dfb admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:49.172463 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d50e99-0c0c-4c2d-8480-2c8a8b286dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:49.172760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d50e99-0c0c-4c2d-8480-2c8a8b286dfb admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:49.173093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:49.173546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d50e99-0c0c-4c2d-8480-2c8a8b286dfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:49.178464 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d50e99-0c0c-4c2d-8480-2c8a8b286dfb admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:49.388022 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 957/1930] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:49 2022] POST /identity/v3/users => generated 290 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:49.392289 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b29c3-f2ff-4576-9464-41d0da4afab7 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:49.405993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b29c3-f2ff-4576-9464-41d0da4afab7 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:49.410905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650be27-2713-4862-ad90-652fbe539448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:49.411563 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650be27-2713-4862-ad90-652fbe539448 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:49.411850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650be27-2713-4862-ad90-652fbe539448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:49.412127 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650be27-2713-4862-ad90-652fbe539448 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:49.412773 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650be27-2713-4862-ad90-652fbe539448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:49.414425 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650be27-2713-4862-ad90-652fbe539448 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:49.426116 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 974/1931] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:49.430405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d50e99-0c0c-4c2d-8480-2c8a8b286dfb admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:49.440894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d50e99-0c0c-4c2d-8480-2c8a8b286dfb admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:49.445553 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842a862-7017-4a5d-8d97-85f4960ebd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:49.446212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842a862-7017-4a5d-8d97-85f4960ebd21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:49.446483 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842a862-7017-4a5d-8d97-85f4960ebd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:49.446758 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842a862-7017-4a5d-8d97-85f4960ebd21 admin admin] PATH_INFO: `/v3/projects/ad42fe1d969342ce954faca5dd671db4/users/25d41f7631ad49c28f080165ba9dfbcf/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:49.466452 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842a862-7017-4a5d-8d97-85f4960ebd21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad42fe1d969342ce954faca5dd671db4, user_id=25d41f7631ad49c28f080165ba9dfbcf, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:49.469249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842a862-7017-4a5d-8d97-85f4960ebd21 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:50.068105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 958/1932] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:49 2022] PUT /identity/v3/projects/ad42fe1d969342ce954faca5dd671db4/users/25d41f7631ad49c28f080165ba9dfbcf/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:25:50.074624 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650be27-2713-4862-ad90-652fbe539448 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:50.091545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650be27-2713-4862-ad90-652fbe539448 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:50.168181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe1d25-9dea-4661-b2f3-3ac015a468a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:50.168970 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe1d25-9dea-4661-b2f3-3ac015a468a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:50.169471 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe1d25-9dea-4661-b2f3-3ac015a468a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:50.169767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe1d25-9dea-4661-b2f3-3ac015a468a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:50.170665 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe1d25-9dea-4661-b2f3-3ac015a468a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:50.172623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe1d25-9dea-4661-b2f3-3ac015a468a9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:50.184535 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 975/1933] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:50.188555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842a862-7017-4a5d-8d97-85f4960ebd21 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:50.205954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842a862-7017-4a5d-8d97-85f4960ebd21 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:50.208992 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9607cb-ed08-404f-9b21-e89f98a0668e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:50.209851 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9607cb-ed08-404f-9b21-e89f98a0668e admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:50.210295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9607cb-ed08-404f-9b21-e89f98a0668e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:50.210685 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9607cb-ed08-404f-9b21-e89f98a0668e admin admin] PATH_INFO: `/v3/projects/ad42fe1d969342ce954faca5dd671db4/users/25d41f7631ad49c28f080165ba9dfbcf/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:50.212925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9607cb-ed08-404f-9b21-e89f98a0668e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad42fe1d969342ce954faca5dd671db4, user_id=25d41f7631ad49c28f080165ba9dfbcf, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:50.216448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9607cb-ed08-404f-9b21-e89f98a0668e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:50.228760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 959/1934] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:50 2022] PUT /identity/v3/projects/ad42fe1d969342ce954faca5dd671db4/users/25d41f7631ad49c28f080165ba9dfbcf/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:50.240311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad0c56-f66e-4f02-aeac-aac870ff7137 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:50.241021 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad0c56-f66e-4f02-aeac-aac870ff7137 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:50.241453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad0c56-f66e-4f02-aeac-aac870ff7137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:50.273959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-86ad0c56-f66e-4f02-aeac-aac870ff7137 None None] MFA Rules not processed for user `25d41f7631ad49c28f080165ba9dfbcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:50.275371 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-86ad0c56-f66e-4f02-aeac-aac870ff7137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:50.357125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 976/1935] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:50.481644 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9607cb-ed08-404f-9b21-e89f98a0668e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:50.483414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-6e9607cb-ed08-404f-9b21-e89f98a0668e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:50.544215 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9607cb-ed08-404f-9b21-e89f98a0668e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:50.549969 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae54b18-c866-4237-92ca-d8c1cbcf6854 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:50.550614 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae54b18-c866-4237-92ca-d8c1cbcf6854 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:50.551017 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae54b18-c866-4237-92ca-d8c1cbcf6854 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:50.551158 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae54b18-c866-4237-92ca-d8c1cbcf6854 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:50.562762 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae54b18-c866-4237-92ca-d8c1cbcf6854 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:50.564303 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae54b18-c866-4237-92ca-d8c1cbcf6854 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:50.573892 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:50.576030 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 960/1936] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:50.716097 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:50.798211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad0c56-f66e-4f02-aeac-aac870ff7137 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:50.808011 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad0c56-f66e-4f02-aeac-aac870ff7137 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:50.812436 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff2263-84f2-4d89-b7c8-7c9614d6d07c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:50.812436 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff2263-84f2-4d89-b7c8-7c9614d6d07c service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:50.812660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff2263-84f2-4d89-b7c8-7c9614d6d07c service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:50.812859 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff2263-84f2-4d89-b7c8-7c9614d6d07c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:50.870729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff2263-84f2-4d89-b7c8-7c9614d6d07c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:50.872224 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff2263-84f2-4d89-b7c8-7c9614d6d07c service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:50.881198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 977/1937] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:50.886523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:50.914801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:51.058203 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:51.298296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:51.549974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:51.652193 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:51.811967 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:51.834804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:52.063505 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:52.109112 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:52.155271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:52.223782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:INSERT=5,keystone:SELECT=80,keystone:DELETE=15 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:52.275019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:52.309368 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:52.331973 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:52.567672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae54b18-c866-4237-92ca-d8c1cbcf6854 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:52.567880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2ae54b18-c866-4237-92ca-d8c1cbcf6854 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:52.629854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:52.633660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae54b18-c866-4237-92ca-d8c1cbcf6854 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:52.638391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9e0e6-a07f-4ea0-ab93-3f2b35ed0926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:52.639031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9e0e6-a07f-4ea0-ab93-3f2b35ed0926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:52.642248 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9e0e6-a07f-4ea0-ab93-3f2b35ed0926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:52.642248 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9e0e6-a07f-4ea0-ab93-3f2b35ed0926 admin admin] PATH_INFO: `/v3/users/bc872356cbba46539164ae0944f950ea` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:52.642248 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9e0e6-a07f-4ea0-ab93-3f2b35ed0926 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc872356cbba46539164ae0944f950ea)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:52.642677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9e0e6-a07f-4ea0-ab93-3f2b35ed0926 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:52.771840 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:52.819943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-2da9e0e6-a07f-4ea0-ab93-3f2b35ed0926 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc872356cbba46539164ae0944f950ea'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:52.834767 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-2da9e0e6-a07f-4ea0-ab93-3f2b35ed0926 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc872356cbba46539164ae0944f950ea'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:52.842651 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-2da9e0e6-a07f-4ea0-ab93-3f2b35ed0926 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc872356cbba46539164ae0944f950ea'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:52.909183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-2da9e0e6-a07f-4ea0-ab93-3f2b35ed0926 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc872356cbba46539164ae0944f950ea'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:52.909932 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-2da9e0e6-a07f-4ea0-ab93-3f2b35ed0926 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc872356cbba46539164ae0944f950ea'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:52.917149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 961/1938] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:52 2022] DELETE /identity/v3/users/bc872356cbba46539164ae0944f950ea => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:52.921659 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed385d6-7056-453f-910c-d0422ce4c170 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:52.921948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed385d6-7056-453f-910c-d0422ce4c170 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:52.922230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed385d6-7056-453f-910c-d0422ce4c170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:52.943219 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-fed385d6-7056-453f-910c-d0422ce4c170 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:52.959822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fed385d6-7056-453f-910c-d0422ce4c170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:52.978331 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:53.007382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 978/1939] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:53.017654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9e0e6-a07f-4ea0-ab93-3f2b35ed0926 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:53.019161 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2da9e0e6-a07f-4ea0-ab93-3f2b35ed0926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:53.081826 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9e0e6-a07f-4ea0-ab93-3f2b35ed0926 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:53.087373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fa32a-e30e-4822-bc3e-9873d98348f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:53.087373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fa32a-e30e-4822-bc3e-9873d98348f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:53.087373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fa32a-e30e-4822-bc3e-9873d98348f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:53.087373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fa32a-e30e-4822-bc3e-9873d98348f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:53.097899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:53.111860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fa32a-e30e-4822-bc3e-9873d98348f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:53.111860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fa32a-e30e-4822-bc3e-9873d98348f7 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:53.121867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:53.126133 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 962/1940] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:53.337800 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:53.394940 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:53.452152 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:53.555281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:53.602171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed385d6-7056-453f-910c-d0422ce4c170 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:53.603687 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fed385d6-7056-453f-910c-d0422ce4c170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:53.613768 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:53.620910 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed385d6-7056-453f-910c-d0422ce4c170 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:53.630148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25496e96-f436-4b0f-b55d-72dd8d0c267d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:53.630943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25496e96-f436-4b0f-b55d-72dd8d0c267d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:53.631388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25496e96-f436-4b0f-b55d-72dd8d0c267d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:53.631650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25496e96-f436-4b0f-b55d-72dd8d0c267d admin admin] PATH_INFO: `/v3/projects/2a214f36bf364c6db3f3507fd93b64f6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:53.634228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25496e96-f436-4b0f-b55d-72dd8d0c267d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a214f36bf364c6db3f3507fd93b64f6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:53.635850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25496e96-f436-4b0f-b55d-72dd8d0c267d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:53.765720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:53.766338 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-25496e96-f436-4b0f-b55d-72dd8d0c267d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a214f36bf364c6db3f3507fd93b64f6', 'request_id': 'req-25496e96-f436-4b0f-b55d-72dd8d0c267d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:53.897231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-25496e96-f436-4b0f-b55d-72dd8d0c267d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a214f36bf364c6db3f3507fd93b64f6', 'request_id': 'req-25496e96-f436-4b0f-b55d-72dd8d0c267d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:53.904172 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-25496e96-f436-4b0f-b55d-72dd8d0c267d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a214f36bf364c6db3f3507fd93b64f6', 'request_id': 'req-25496e96-f436-4b0f-b55d-72dd8d0c267d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:53.914660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-25496e96-f436-4b0f-b55d-72dd8d0c267d admin admin] The token cache is being invalidate because project 2a214f36bf364c6db3f3507fd93b64f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:53.914937 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-25496e96-f436-4b0f-b55d-72dd8d0c267d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:53.915610 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-25496e96-f436-4b0f-b55d-72dd8d0c267d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:53.917096 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 979/1941] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:53 2022] DELETE /identity/v3/projects/2a214f36bf364c6db3f3507fd93b64f6 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:53.921184 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fa32a-e30e-4822-bc3e-9873d98348f7 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:53.931213 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fa32a-e30e-4822-bc3e-9873d98348f7 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:53.972272 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:53.991448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced32eaf-5cf1-4d2d-ba72-85278d114c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:53.992103 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced32eaf-5cf1-4d2d-ba72-85278d114c87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:53.992504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced32eaf-5cf1-4d2d-ba72-85278d114c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:53.992788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced32eaf-5cf1-4d2d-ba72-85278d114c87 admin admin] PATH_INFO: `/v3/users/79fdb72e792648ad9e684403f97b0f0a` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:53.994002 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced32eaf-5cf1-4d2d-ba72-85278d114c87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79fdb72e792648ad9e684403f97b0f0a)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:53.996486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced32eaf-5cf1-4d2d-ba72-85278d114c87 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:54.058271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:54.193611 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ced32eaf-5cf1-4d2d-ba72-85278d114c87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79fdb72e792648ad9e684403f97b0f0a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:54.215697 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ced32eaf-5cf1-4d2d-ba72-85278d114c87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79fdb72e792648ad9e684403f97b0f0a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:54.226155 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ced32eaf-5cf1-4d2d-ba72-85278d114c87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79fdb72e792648ad9e684403f97b0f0a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:54.343114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ced32eaf-5cf1-4d2d-ba72-85278d114c87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79fdb72e792648ad9e684403f97b0f0a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:54.343806 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ced32eaf-5cf1-4d2d-ba72-85278d114c87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79fdb72e792648ad9e684403f97b0f0a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:54.350583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 963/1942] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:53 2022] DELETE /identity/v3/users/79fdb72e792648ad9e684403f97b0f0a => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:54.372974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:54.669881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:54.709244 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:54.785379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25496e96-f436-4b0f-b55d-72dd8d0c267d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:54.790110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-25496e96-f436-4b0f-b55d-72dd8d0c267d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:54.800771 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25496e96-f436-4b0f-b55d-72dd8d0c267d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:54.865062 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e2937-bd76-48eb-a123-3807d033301d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:54.870380 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e2937-bd76-48eb-a123-3807d033301d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:54.870380 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e2937-bd76-48eb-a123-3807d033301d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:54.870380 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e2937-bd76-48eb-a123-3807d033301d admin admin] PATH_INFO: `/v3/projects/7ce4a0f15b344cff891072b691030897` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:54.870380 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e2937-bd76-48eb-a123-3807d033301d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ce4a0f15b344cff891072b691030897)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:54.870380 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e2937-bd76-48eb-a123-3807d033301d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:54.880320 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:54.995274 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6e2e2937-bd76-48eb-a123-3807d033301d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ce4a0f15b344cff891072b691030897', 'request_id': 'req-6e2e2937-bd76-48eb-a123-3807d033301d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:55.023428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:55.063112 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6e2e2937-bd76-48eb-a123-3807d033301d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ce4a0f15b344cff891072b691030897', 'request_id': 'req-6e2e2937-bd76-48eb-a123-3807d033301d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:55.069942 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6e2e2937-bd76-48eb-a123-3807d033301d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ce4a0f15b344cff891072b691030897', 'request_id': 'req-6e2e2937-bd76-48eb-a123-3807d033301d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:55.081304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6e2e2937-bd76-48eb-a123-3807d033301d admin admin] The token cache is being invalidate because project 7ce4a0f15b344cff891072b691030897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:55.081600 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6e2e2937-bd76-48eb-a123-3807d033301d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:55.082264 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6e2e2937-bd76-48eb-a123-3807d033301d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:55.083896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 980/1943] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:54 2022] DELETE /identity/v3/projects/7ce4a0f15b344cff891072b691030897 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:55.089356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879bf90-f672-471a-afb2-d1570df9c93a None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:55.089745 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879bf90-f672-471a-afb2-d1570df9c93a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:55.089990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879bf90-f672-471a-afb2-d1570df9c93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:55.115393 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-4879bf90-f672-471a-afb2-d1570df9c93a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:55.116906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4879bf90-f672-471a-afb2-d1570df9c93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:55.167862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:55.183618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 964/1944] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:55.192657 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07ab56-ce5b-4dd3-87fd-057f76db9f29 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:55.193048 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07ab56-ce5b-4dd3-87fd-057f76db9f29 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:55.202580 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07ab56-ce5b-4dd3-87fd-057f76db9f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:55.215549 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-7f07ab56-ce5b-4dd3-87fd-057f76db9f29 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:55.216912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7f07ab56-ce5b-4dd3-87fd-057f76db9f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:55.228673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 981/1945] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:55.234660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879bf90-f672-471a-afb2-d1570df9c93a None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:55.269817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879bf90-f672-471a-afb2-d1570df9c93a None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:55.282868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405efa6-1f0a-47fd-a3d4-12534aa47d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:55.282868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405efa6-1f0a-47fd-a3d4-12534aa47d06 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:55.282868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405efa6-1f0a-47fd-a3d4-12534aa47d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:55.282868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405efa6-1f0a-47fd-a3d4-12534aa47d06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:55.283344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405efa6-1f0a-47fd-a3d4-12534aa47d06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:55.288286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405efa6-1f0a-47fd-a3d4-12534aa47d06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:55.289563 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405efa6-1f0a-47fd-a3d4-12534aa47d06 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:55.306459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 965/1946] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:25:55.308246 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07ab56-ce5b-4dd3-87fd-057f76db9f29 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:55.319372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07ab56-ce5b-4dd3-87fd-057f76db9f29 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:55.324195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba2fa9-8fc4-486d-a889-66b99b290c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:55.324819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba2fa9-8fc4-486d-a889-66b99b290c0d admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:55.325102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba2fa9-8fc4-486d-a889-66b99b290c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:55.325378 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba2fa9-8fc4-486d-a889-66b99b290c0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:55.326022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba2fa9-8fc4-486d-a889-66b99b290c0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:55.327969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba2fa9-8fc4-486d-a889-66b99b290c0d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:55.361049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:55.429643 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 982/1947] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:55 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:55.433281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405efa6-1f0a-47fd-a3d4-12534aa47d06 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:55.443858 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405efa6-1f0a-47fd-a3d4-12534aa47d06 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:55.522339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4937b6-6203-4dfd-98ad-486fb56d24f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:55.522956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4937b6-6203-4dfd-98ad-486fb56d24f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:55.523277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4937b6-6203-4dfd-98ad-486fb56d24f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:55.524808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4937b6-6203-4dfd-98ad-486fb56d24f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:55.524808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4937b6-6203-4dfd-98ad-486fb56d24f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:55.526771 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4937b6-6203-4dfd-98ad-486fb56d24f1 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:55.601512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 966/1948] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:55 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:55.605693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba2fa9-8fc4-486d-a889-66b99b290c0d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:55.616630 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba2fa9-8fc4-486d-a889-66b99b290c0d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:55.621384 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b265a1e-5d13-4cea-8490-28edd879d36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:55.622274 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b265a1e-5d13-4cea-8490-28edd879d36c admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:55.622274 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b265a1e-5d13-4cea-8490-28edd879d36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:55.622611 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b265a1e-5d13-4cea-8490-28edd879d36c admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:55.623303 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b265a1e-5d13-4cea-8490-28edd879d36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:55.624918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b265a1e-5d13-4cea-8490-28edd879d36c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:55.635431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 983/1949] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:25:55.639766 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4937b6-6203-4dfd-98ad-486fb56d24f1 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:55.649915 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4937b6-6203-4dfd-98ad-486fb56d24f1 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:55.655478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb8b78-23a5-4a24-98ee-3a5269508ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:55.656171 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb8b78-23a5-4a24-98ee-3a5269508ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:55.656438 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb8b78-23a5-4a24-98ee-3a5269508ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:55.656677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb8b78-23a5-4a24-98ee-3a5269508ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:55.657322 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb8b78-23a5-4a24-98ee-3a5269508ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:55.658931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb8b78-23a5-4a24-98ee-3a5269508ee5 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:55.663017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:55.671763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 967/1950] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:55.675550 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b265a1e-5d13-4cea-8490-28edd879d36c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:55.686199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b265a1e-5d13-4cea-8490-28edd879d36c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:55.690870 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a842a5a-d7b3-4a84-814b-14ab9fc99bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:55.691603 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a842a5a-d7b3-4a84-814b-14ab9fc99bea admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:55.691865 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a842a5a-d7b3-4a84-814b-14ab9fc99bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:55.692137 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a842a5a-d7b3-4a84-814b-14ab9fc99bea admin admin] PATH_INFO: `/v3/projects/25fe0c1718004cc195bfad518ce50b67/users/e0f0a526d8214180bd60ef4bf2ce4b56/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:55.698197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:55.716390 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a842a5a-d7b3-4a84-814b-14ab9fc99bea admin admin] RBAC: Authorizing `identity:create_grant(project_id=25fe0c1718004cc195bfad518ce50b67, user_id=e0f0a526d8214180bd60ef4bf2ce4b56, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:55.720953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a842a5a-d7b3-4a84-814b-14ab9fc99bea admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:55.772699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 984/1951] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:55 2022] PUT /identity/v3/projects/25fe0c1718004cc195bfad518ce50b67/users/e0f0a526d8214180bd60ef4bf2ce4b56/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:55.784356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd1d7d-2dba-4bba-8094-5f29d741728e None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:55.784356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd1d7d-2dba-4bba-8094-5f29d741728e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:55.784533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd1d7d-2dba-4bba-8094-5f29d741728e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:55.817293 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-f0cd1d7d-2dba-4bba-8094-5f29d741728e None None] MFA Rules not processed for user `e0f0a526d8214180bd60ef4bf2ce4b56`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:55.818663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f0cd1d7d-2dba-4bba-8094-5f29d741728e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:55.888680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 968/1952] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:55.893395 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:55.910522 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a842a5a-d7b3-4a84-814b-14ab9fc99bea admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:55.910522 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0a842a5a-d7b3-4a84-814b-14ab9fc99bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:55.987090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a842a5a-d7b3-4a84-814b-14ab9fc99bea admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:55.995152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1842965-0920-467f-b3fb-c65290118a96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:55.996214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1842965-0920-467f-b3fb-c65290118a96 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:55.996214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1842965-0920-467f-b3fb-c65290118a96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:55.996835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1842965-0920-467f-b3fb-c65290118a96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:56.005440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1842965-0920-467f-b3fb-c65290118a96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:56.006860 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1842965-0920-467f-b3fb-c65290118a96 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:56.016916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 985/1953] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:56.036014 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd1d7d-2dba-4bba-8094-5f29d741728e None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:56.102677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd1d7d-2dba-4bba-8094-5f29d741728e None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:56.107579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f3236-eb32-4b1e-bc58-a8c1b7b23236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:56.108271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f3236-eb32-4b1e-bc58-a8c1b7b23236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:56.108545 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f3236-eb32-4b1e-bc58-a8c1b7b23236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:56.108814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f3236-eb32-4b1e-bc58-a8c1b7b23236 admin admin] PATH_INFO: `/v3/users/e0f0a526d8214180bd60ef4bf2ce4b56` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:56.110060 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f3236-eb32-4b1e-bc58-a8c1b7b23236 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0f0a526d8214180bd60ef4bf2ce4b56)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:56.116966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f3236-eb32-4b1e-bc58-a8c1b7b23236 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:56.254100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:56.334153 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e52f3236-eb32-4b1e-bc58-a8c1b7b23236 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0f0a526d8214180bd60ef4bf2ce4b56'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:56.346229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e52f3236-eb32-4b1e-bc58-a8c1b7b23236 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0f0a526d8214180bd60ef4bf2ce4b56'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:56.352939 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e52f3236-eb32-4b1e-bc58-a8c1b7b23236 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0f0a526d8214180bd60ef4bf2ce4b56'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:56.400184 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e52f3236-eb32-4b1e-bc58-a8c1b7b23236 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0f0a526d8214180bd60ef4bf2ce4b56'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:56.400733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e52f3236-eb32-4b1e-bc58-a8c1b7b23236 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0f0a526d8214180bd60ef4bf2ce4b56'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:56.407878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 969/1954] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:56 2022] DELETE /identity/v3/users/e0f0a526d8214180bd60ef4bf2ce4b56 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:56.412506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0030f-c4ee-4dec-9433-d79fabe5b5a4 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:56.412795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0030f-c4ee-4dec-9433-d79fabe5b5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:56.413147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0030f-c4ee-4dec-9433-d79fabe5b5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:56.434426 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c0a0030f-c4ee-4dec-9433-d79fabe5b5a4 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:56.435694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c0a0030f-c4ee-4dec-9433-d79fabe5b5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:56.496578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 986/1955] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:56.505863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f3236-eb32-4b1e-bc58-a8c1b7b23236 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:56.507440 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e52f3236-eb32-4b1e-bc58-a8c1b7b23236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:56.555268 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:56.569233 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f3236-eb32-4b1e-bc58-a8c1b7b23236 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:56.573047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4ad95-42fc-4112-8a0b-ff1c6568b1b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:56.573668 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4ad95-42fc-4112-8a0b-ff1c6568b1b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:56.573950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4ad95-42fc-4112-8a0b-ff1c6568b1b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:56.574223 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4ad95-42fc-4112-8a0b-ff1c6568b1b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:56.583816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4ad95-42fc-4112-8a0b-ff1c6568b1b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:56.585258 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4ad95-42fc-4112-8a0b-ff1c6568b1b7 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:56.595937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 970/1956] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:57.152529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0030f-c4ee-4dec-9433-d79fabe5b5a4 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:57.154133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c0a0030f-c4ee-4dec-9433-d79fabe5b5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:57.167686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0030f-c4ee-4dec-9433-d79fabe5b5a4 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:57.172361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c2e0a-a556-4ab6-accb-0f6ad31e9ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:57.173017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c2e0a-a556-4ab6-accb-0f6ad31e9ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:57.173297 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c2e0a-a556-4ab6-accb-0f6ad31e9ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:57.173567 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c2e0a-a556-4ab6-accb-0f6ad31e9ab9 admin admin] PATH_INFO: `/v3/projects/25fe0c1718004cc195bfad518ce50b67` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:57.174727 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c2e0a-a556-4ab6-accb-0f6ad31e9ab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25fe0c1718004cc195bfad518ce50b67)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:57.176670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c2e0a-a556-4ab6-accb-0f6ad31e9ab9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:57.291973 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d85c2e0a-a556-4ab6-accb-0f6ad31e9ab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25fe0c1718004cc195bfad518ce50b67', 'request_id': 'req-d85c2e0a-a556-4ab6-accb-0f6ad31e9ab9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:57.342973 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d85c2e0a-a556-4ab6-accb-0f6ad31e9ab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25fe0c1718004cc195bfad518ce50b67', 'request_id': 'req-d85c2e0a-a556-4ab6-accb-0f6ad31e9ab9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:57.350036 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d85c2e0a-a556-4ab6-accb-0f6ad31e9ab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25fe0c1718004cc195bfad518ce50b67', 'request_id': 'req-d85c2e0a-a556-4ab6-accb-0f6ad31e9ab9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:57.357678 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:57.362376 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d85c2e0a-a556-4ab6-accb-0f6ad31e9ab9 admin admin] The token cache is being invalidate because project 25fe0c1718004cc195bfad518ce50b67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:57.362668 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d85c2e0a-a556-4ab6-accb-0f6ad31e9ab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:57.367366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d85c2e0a-a556-4ab6-accb-0f6ad31e9ab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:57.367366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 987/1957] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:57 2022] DELETE /identity/v3/projects/25fe0c1718004cc195bfad518ce50b67 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:57.369753 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56912328-8489-42ca-ab65-4ea4511990bf None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:57.370109 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56912328-8489-42ca-ab65-4ea4511990bf None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:57.370441 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56912328-8489-42ca-ab65-4ea4511990bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:57.392532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-56912328-8489-42ca-ab65-4ea4511990bf None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:57.392532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-56912328-8489-42ca-ab65-4ea4511990bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:57.462507 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 971/1958] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:57.471039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f169f-afd4-4636-9bdf-99019da0b295 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:57.471372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f169f-afd4-4636-9bdf-99019da0b295 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:57.471645 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f169f-afd4-4636-9bdf-99019da0b295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:57.492006 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-9d5f169f-afd4-4636-9bdf-99019da0b295 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:57.493212 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9d5f169f-afd4-4636-9bdf-99019da0b295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:57.501427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 988/1959] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:57.505740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56912328-8489-42ca-ab65-4ea4511990bf None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:57.515551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56912328-8489-42ca-ab65-4ea4511990bf None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:57.520124 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d39565-d05c-4f74-bf04-451f31cb91a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:57.520726 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d39565-d05c-4f74-bf04-451f31cb91a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:57.521007 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d39565-d05c-4f74-bf04-451f31cb91a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:57.521279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d39565-d05c-4f74-bf04-451f31cb91a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:57.521923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d39565-d05c-4f74-bf04-451f31cb91a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:57.522225 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d39565-d05c-4f74-bf04-451f31cb91a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:57.523849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d39565-d05c-4f74-bf04-451f31cb91a7 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:57.535955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 972/1960] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:25:57.540259 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f169f-afd4-4636-9bdf-99019da0b295 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:57.550171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f169f-afd4-4636-9bdf-99019da0b295 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:57.554791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0778a76-890f-4a84-8c8e-81b6865f2bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:57.555400 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0778a76-890f-4a84-8c8e-81b6865f2bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:57.555676 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0778a76-890f-4a84-8c8e-81b6865f2bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:57.555944 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0778a76-890f-4a84-8c8e-81b6865f2bf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:57.556576 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0778a76-890f-4a84-8c8e-81b6865f2bf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:57.558461 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0778a76-890f-4a84-8c8e-81b6865f2bf2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:57.653756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 989/1961] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:57 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:57.658172 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d39565-d05c-4f74-bf04-451f31cb91a7 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:57.668925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d39565-d05c-4f74-bf04-451f31cb91a7 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:57.699966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:57.727395 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657a6ae-a89f-4dc8-aaed-68bbd658ad87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:57.727924 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657a6ae-a89f-4dc8-aaed-68bbd658ad87 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:57.728202 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657a6ae-a89f-4dc8-aaed-68bbd658ad87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:57.728472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657a6ae-a89f-4dc8-aaed-68bbd658ad87 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:57.729160 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657a6ae-a89f-4dc8-aaed-68bbd658ad87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:57.731224 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657a6ae-a89f-4dc8-aaed-68bbd658ad87 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:57.836920 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 973/1962] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:57 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:57.841420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0778a76-890f-4a84-8c8e-81b6865f2bf2 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:57.856070 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0778a76-890f-4a84-8c8e-81b6865f2bf2 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:57.861168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c765b3-2170-43e5-b1d1-55360c573737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:57.861737 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c765b3-2170-43e5-b1d1-55360c573737 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:57.862020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c765b3-2170-43e5-b1d1-55360c573737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:57.862305 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c765b3-2170-43e5-b1d1-55360c573737 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:57.862964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c765b3-2170-43e5-b1d1-55360c573737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:57.864637 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c765b3-2170-43e5-b1d1-55360c573737 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:57.882589 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 990/1963] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:57.887492 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657a6ae-a89f-4dc8-aaed-68bbd658ad87 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:57.897735 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657a6ae-a89f-4dc8-aaed-68bbd658ad87 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:57.903312 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece1a05-d67a-4d2d-8c45-3935b9e12226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:57.904052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece1a05-d67a-4d2d-8c45-3935b9e12226 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:57.904542 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece1a05-d67a-4d2d-8c45-3935b9e12226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:57.904885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece1a05-d67a-4d2d-8c45-3935b9e12226 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:57.905609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece1a05-d67a-4d2d-8c45-3935b9e12226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:57.907253 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece1a05-d67a-4d2d-8c45-3935b9e12226 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:57.918667 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 974/1964] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:57.925115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c765b3-2170-43e5-b1d1-55360c573737 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:57.936808 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c765b3-2170-43e5-b1d1-55360c573737 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:57.941600 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a02441-8bf8-4349-8f4b-403e834ffc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:57.942271 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a02441-8bf8-4349-8f4b-403e834ffc0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:57.942555 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a02441-8bf8-4349-8f4b-403e834ffc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:57.942836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a02441-8bf8-4349-8f4b-403e834ffc0a admin admin] PATH_INFO: `/v3/projects/3740f34b64804368b6f8a864482ac16f/users/1d93d5b4f18d48fa923fccf21b2bd111/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:57.959083 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a02441-8bf8-4349-8f4b-403e834ffc0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3740f34b64804368b6f8a864482ac16f, user_id=1d93d5b4f18d48fa923fccf21b2bd111, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:57.962079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a02441-8bf8-4349-8f4b-403e834ffc0a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:58.023464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 991/1965] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:57 2022] PUT /identity/v3/projects/3740f34b64804368b6f8a864482ac16f/users/1d93d5b4f18d48fa923fccf21b2bd111/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:58.035039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcb2e7-ac54-4b10-8052-d4c02bb4ab47 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:58.035356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcb2e7-ac54-4b10-8052-d4c02bb4ab47 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:58.035628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcb2e7-ac54-4b10-8052-d4c02bb4ab47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:58.084211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-72bcb2e7-ac54-4b10-8052-d4c02bb4ab47 None None] MFA Rules not processed for user `1d93d5b4f18d48fa923fccf21b2bd111`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:58.084211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-72bcb2e7-ac54-4b10-8052-d4c02bb4ab47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:58.149108 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 975/1966] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:58.160415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a02441-8bf8-4349-8f4b-403e834ffc0a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:58.162251 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-69a02441-8bf8-4349-8f4b-403e834ffc0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:58.219566 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a02441-8bf8-4349-8f4b-403e834ffc0a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:58.223292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b18526-8f41-44b9-bdd6-478e0b6d1d3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:58.223902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b18526-8f41-44b9-bdd6-478e0b6d1d3f service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:58.224181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b18526-8f41-44b9-bdd6-478e0b6d1d3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:58.224453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b18526-8f41-44b9-bdd6-478e0b6d1d3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:58.232776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b18526-8f41-44b9-bdd6-478e0b6d1d3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:58.234180 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b18526-8f41-44b9-bdd6-478e0b6d1d3f service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:58.243076 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 992/1967] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:58.265755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcb2e7-ac54-4b10-8052-d4c02bb4ab47 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:58.327188 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcb2e7-ac54-4b10-8052-d4c02bb4ab47 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:58.332034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfa52b-5c79-4bcb-9221-ad105668edfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:58.332669 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfa52b-5c79-4bcb-9221-ad105668edfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:58.332947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfa52b-5c79-4bcb-9221-ad105668edfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:58.333218 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfa52b-5c79-4bcb-9221-ad105668edfe admin admin] PATH_INFO: `/v3/users/1d93d5b4f18d48fa923fccf21b2bd111` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:58.334343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bfa52b-5c79-4bcb-9221-ad105668edfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d93d5b4f18d48fa923fccf21b2bd111)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:58.336344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bfa52b-5c79-4bcb-9221-ad105668edfe admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:58.538053 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:58.541428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-44bfa52b-5c79-4bcb-9221-ad105668edfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d93d5b4f18d48fa923fccf21b2bd111'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:58.561376 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-44bfa52b-5c79-4bcb-9221-ad105668edfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d93d5b4f18d48fa923fccf21b2bd111'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:58.568372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-44bfa52b-5c79-4bcb-9221-ad105668edfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d93d5b4f18d48fa923fccf21b2bd111'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:58.686145 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-44bfa52b-5c79-4bcb-9221-ad105668edfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d93d5b4f18d48fa923fccf21b2bd111'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:58.686824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-44bfa52b-5c79-4bcb-9221-ad105668edfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d93d5b4f18d48fa923fccf21b2bd111'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:58.693684 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 976/1968] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:58 2022] DELETE /identity/v3/users/1d93d5b4f18d48fa923fccf21b2bd111 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:58.698533 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707d4f7-9d5d-42d4-ab4a-6acc04f9b86c None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:58.698909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707d4f7-9d5d-42d4-ab4a-6acc04f9b86c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:58.699111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707d4f7-9d5d-42d4-ab4a-6acc04f9b86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:58.720526 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-1707d4f7-9d5d-42d4-ab4a-6acc04f9b86c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:58.722073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1707d4f7-9d5d-42d4-ab4a-6acc04f9b86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:58.791800 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 993/1969] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:58.802355 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfa52b-5c79-4bcb-9221-ad105668edfe admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:58.804083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-44bfa52b-5c79-4bcb-9221-ad105668edfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:58.862231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfa52b-5c79-4bcb-9221-ad105668edfe admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:58.865901 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797553a-2c00-41b8-9073-2bc170a7f0ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:58.866510 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797553a-2c00-41b8-9073-2bc170a7f0ea service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:58.866786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797553a-2c00-41b8-9073-2bc170a7f0ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:58.867062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797553a-2c00-41b8-9073-2bc170a7f0ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:58.876332 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797553a-2c00-41b8-9073-2bc170a7f0ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:58.877834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797553a-2c00-41b8-9073-2bc170a7f0ea service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:58.888315 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 977/1970] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:25:59.551622 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707d4f7-9d5d-42d4-ab4a-6acc04f9b86c None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:59.553606 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1707d4f7-9d5d-42d4-ab4a-6acc04f9b86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:59.567832 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707d4f7-9d5d-42d4-ab4a-6acc04f9b86c None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:59.572515 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3a48f-1551-4b60-be2c-40cb5e330cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:59.573161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3a48f-1551-4b60-be2c-40cb5e330cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:59.573445 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3a48f-1551-4b60-be2c-40cb5e330cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:59.573717 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3a48f-1551-4b60-be2c-40cb5e330cf4 admin admin] PATH_INFO: `/v3/projects/3740f34b64804368b6f8a864482ac16f` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:59.575496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3a48f-1551-4b60-be2c-40cb5e330cf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3740f34b64804368b6f8a864482ac16f)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:59.583152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3a48f-1551-4b60-be2c-40cb5e330cf4 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:59.708089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:59.722458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-62a3a48f-1551-4b60-be2c-40cb5e330cf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3740f34b64804368b6f8a864482ac16f', 'request_id': 'req-62a3a48f-1551-4b60-be2c-40cb5e330cf4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:59.800218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-62a3a48f-1551-4b60-be2c-40cb5e330cf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3740f34b64804368b6f8a864482ac16f', 'request_id': 'req-62a3a48f-1551-4b60-be2c-40cb5e330cf4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:59.818837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-62a3a48f-1551-4b60-be2c-40cb5e330cf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3740f34b64804368b6f8a864482ac16f', 'request_id': 'req-62a3a48f-1551-4b60-be2c-40cb5e330cf4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:59.833330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-62a3a48f-1551-4b60-be2c-40cb5e330cf4 admin admin] The token cache is being invalidate because project 3740f34b64804368b6f8a864482ac16f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:59.833887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-62a3a48f-1551-4b60-be2c-40cb5e330cf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:59.834761 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-62a3a48f-1551-4b60-be2c-40cb5e330cf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:59.837323 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 994/1971] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:59 2022] DELETE /identity/v3/projects/3740f34b64804368b6f8a864482ac16f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:59.851256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79064e28-69b6-4150-9260-a8cd5b8b5d38 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:59.851785 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79064e28-69b6-4150-9260-a8cd5b8b5d38 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:59.851927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79064e28-69b6-4150-9260-a8cd5b8b5d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:59.873084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-79064e28-69b6-4150-9260-a8cd5b8b5d38 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:59.874311 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-79064e28-69b6-4150-9260-a8cd5b8b5d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:59.935812 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 978/1972] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:59.943662 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375b40d-8d17-4f63-8983-3b4395f22aa6 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:59.943955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375b40d-8d17-4f63-8983-3b4395f22aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:59.944234 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375b40d-8d17-4f63-8983-3b4395f22aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:59.965228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-7375b40d-8d17-4f63-8983-3b4395f22aa6 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:59.967343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7375b40d-8d17-4f63-8983-3b4395f22aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:59.976133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 995/1973] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:59.981240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79064e28-69b6-4150-9260-a8cd5b8b5d38 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:59.995038 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79064e28-69b6-4150-9260-a8cd5b8b5d38 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:00.000711 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3977f7-4b10-4067-ad06-268acca33783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:00.001576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3977f7-4b10-4067-ad06-268acca33783 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:00.001779 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3977f7-4b10-4067-ad06-268acca33783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:00.002057 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3977f7-4b10-4067-ad06-268acca33783 admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:00.002864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3977f7-4b10-4067-ad06-268acca33783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:26:00.003183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3977f7-4b10-4067-ad06-268acca33783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:00.004811 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3977f7-4b10-4067-ad06-268acca33783 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:00.025491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375b40d-8d17-4f63-8983-3b4395f22aa6 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:00.027503 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 979/1974] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:26:00.036760 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375b40d-8d17-4f63-8983-3b4395f22aa6 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:00.041714 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8ddbd-799b-46d6-95a5-0a0167d90d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:00.042335 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8ddbd-799b-46d6-95a5-0a0167d90d9e admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:00.042813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8ddbd-799b-46d6-95a5-0a0167d90d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:00.043732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8ddbd-799b-46d6-95a5-0a0167d90d9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:00.044404 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8ddbd-799b-46d6-95a5-0a0167d90d9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:00.046534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8ddbd-799b-46d6-95a5-0a0167d90d9e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:00.229454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 996/1975] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:00 2022] POST /identity/v3/projects => generated 359 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:00.234808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3977f7-4b10-4067-ad06-268acca33783 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:00.245715 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3977f7-4b10-4067-ad06-268acca33783 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:00.297076 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:00.306624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1b990-5d20-4189-9153-b9fe2cbbddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:00.307272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1b990-5d20-4189-9153-b9fe2cbbddf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:00.307551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1b990-5d20-4189-9153-b9fe2cbbddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:00.307825 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1b990-5d20-4189-9153-b9fe2cbbddf1 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:00.308450 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1b990-5d20-4189-9153-b9fe2cbbddf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:00.313359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1b990-5d20-4189-9153-b9fe2cbbddf1 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:00.407058 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 980/1976] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:00 2022] POST /identity/v3/users => generated 280 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:00.411310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8ddbd-799b-46d6-95a5-0a0167d90d9e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:00.422224 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8ddbd-799b-46d6-95a5-0a0167d90d9e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:00.426941 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e109a5-782d-4510-b7f5-62228411ea2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:00.427595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e109a5-782d-4510-b7f5-62228411ea2f admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:00.427875 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e109a5-782d-4510-b7f5-62228411ea2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:00.428149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e109a5-782d-4510-b7f5-62228411ea2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:00.428796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e109a5-782d-4510-b7f5-62228411ea2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:00.430392 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e109a5-782d-4510-b7f5-62228411ea2f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:00.446699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 997/1977] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:00.451163 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1b990-5d20-4189-9153-b9fe2cbbddf1 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:00.462496 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1b990-5d20-4189-9153-b9fe2cbbddf1 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:00.467713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e1bb8-20d1-48c2-b48e-46af630d08be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:00.468321 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e1bb8-20d1-48c2-b48e-46af630d08be admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:00.468601 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e1bb8-20d1-48c2-b48e-46af630d08be admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:00.468875 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e1bb8-20d1-48c2-b48e-46af630d08be admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:00.469513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e1bb8-20d1-48c2-b48e-46af630d08be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:00.471104 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e1bb8-20d1-48c2-b48e-46af630d08be admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:00.485562 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 981/1978] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:00.489271 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e109a5-782d-4510-b7f5-62228411ea2f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:00.498846 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e109a5-782d-4510-b7f5-62228411ea2f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:00.503868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4468d1c-90c8-4ab7-be57-8d16ac926cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:00.504638 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4468d1c-90c8-4ab7-be57-8d16ac926cca admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:00.504994 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4468d1c-90c8-4ab7-be57-8d16ac926cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:00.505335 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4468d1c-90c8-4ab7-be57-8d16ac926cca admin admin] PATH_INFO: `/v3/projects/ee63d4aa1eb84493a6973e76c453efe6/users/cea150d0285b44c38f64c7ab530711f2/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:00.522318 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4468d1c-90c8-4ab7-be57-8d16ac926cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee63d4aa1eb84493a6973e76c453efe6, user_id=cea150d0285b44c38f64c7ab530711f2, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:00.525065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4468d1c-90c8-4ab7-be57-8d16ac926cca admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:00.606823 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 998/1979] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:00 2022] PUT /identity/v3/projects/ee63d4aa1eb84493a6973e76c453efe6/users/cea150d0285b44c38f64c7ab530711f2/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:26:00.617322 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae66b4f-1f4e-4fe7-92d5-ef9f67170783 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:00.617608 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae66b4f-1f4e-4fe7-92d5-ef9f67170783 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:00.617966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae66b4f-1f4e-4fe7-92d5-ef9f67170783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:00.647576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-bae66b4f-1f4e-4fe7-92d5-ef9f67170783 None None] MFA Rules not processed for user `cea150d0285b44c38f64c7ab530711f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:00.648821 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-bae66b4f-1f4e-4fe7-92d5-ef9f67170783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:00.720223 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 982/1980] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:00 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:00.766486 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4468d1c-90c8-4ab7-be57-8d16ac926cca admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:00.770134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d4468d1c-90c8-4ab7-be57-8d16ac926cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:00.842592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4468d1c-90c8-4ab7-be57-8d16ac926cca admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:00.846412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a89ad-96d7-4122-aeae-74c78963cb88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:00.847106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a89ad-96d7-4122-aeae-74c78963cb88 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:00.847480 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a89ad-96d7-4122-aeae-74c78963cb88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:00.847819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a89ad-96d7-4122-aeae-74c78963cb88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:00.866699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a89ad-96d7-4122-aeae-74c78963cb88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:00.869915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a89ad-96d7-4122-aeae-74c78963cb88 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:00.884144 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 999/1981] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:00 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:00.898715 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae66b4f-1f4e-4fe7-92d5-ef9f67170783 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:00.990662 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae66b4f-1f4e-4fe7-92d5-ef9f67170783 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:00.998141 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710f600-3679-4a33-9a22-46d3dcafc62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:00.999072 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710f600-3679-4a33-9a22-46d3dcafc62f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:01.005488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710f600-3679-4a33-9a22-46d3dcafc62f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:01.005488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710f600-3679-4a33-9a22-46d3dcafc62f admin admin] PATH_INFO: `/v3/users/cea150d0285b44c38f64c7ab530711f2` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:01.005488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710f600-3679-4a33-9a22-46d3dcafc62f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cea150d0285b44c38f64c7ab530711f2)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:01.005488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710f600-3679-4a33-9a22-46d3dcafc62f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:01.190037 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e710f600-3679-4a33-9a22-46d3dcafc62f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cea150d0285b44c38f64c7ab530711f2'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:01.219999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:01.219999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e710f600-3679-4a33-9a22-46d3dcafc62f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cea150d0285b44c38f64c7ab530711f2'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:01.219999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e710f600-3679-4a33-9a22-46d3dcafc62f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cea150d0285b44c38f64c7ab530711f2'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:01.368155 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e710f600-3679-4a33-9a22-46d3dcafc62f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cea150d0285b44c38f64c7ab530711f2'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:01.369124 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e710f600-3679-4a33-9a22-46d3dcafc62f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cea150d0285b44c38f64c7ab530711f2'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:01.384475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 983/1982] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:00 2022] DELETE /identity/v3/users/cea150d0285b44c38f64c7ab530711f2 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:01.385454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dc66b-6d81-4b11-b729-08404facba0a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:01.385802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dc66b-6d81-4b11-b729-08404facba0a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:01.386128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dc66b-6d81-4b11-b729-08404facba0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:01.413735 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-3c7dc66b-6d81-4b11-b729-08404facba0a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:01.416382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3c7dc66b-6d81-4b11-b729-08404facba0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:01.493867 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:01.504525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1000/1983] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:01.517929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710f600-3679-4a33-9a22-46d3dcafc62f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:01.519539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e710f600-3679-4a33-9a22-46d3dcafc62f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:01.578656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710f600-3679-4a33-9a22-46d3dcafc62f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:01.582400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2763b5-33c0-4649-8684-8a26e029cd4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:01.583012 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2763b5-33c0-4649-8684-8a26e029cd4e service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:01.591443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2763b5-33c0-4649-8684-8a26e029cd4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:01.591443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2763b5-33c0-4649-8684-8a26e029cd4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:01.594079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2763b5-33c0-4649-8684-8a26e029cd4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:01.596210 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2763b5-33c0-4649-8684-8a26e029cd4e service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:01.606138 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 984/1984] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:26:02.139475 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7dc66b-6d81-4b11-b729-08404facba0a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:02.141028 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3c7dc66b-6d81-4b11-b729-08404facba0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:02.156652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7dc66b-6d81-4b11-b729-08404facba0a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:02.164801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ecef7e-50a6-4669-a94b-9de12677f0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:02.168059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecef7e-50a6-4669-a94b-9de12677f0b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:02.168351 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecef7e-50a6-4669-a94b-9de12677f0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:02.168629 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecef7e-50a6-4669-a94b-9de12677f0b9 admin admin] PATH_INFO: `/v3/projects/ee63d4aa1eb84493a6973e76c453efe6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:02.174519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ecef7e-50a6-4669-a94b-9de12677f0b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee63d4aa1eb84493a6973e76c453efe6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:02.174519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ecef7e-50a6-4669-a94b-9de12677f0b9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:02.346531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-31ecef7e-50a6-4669-a94b-9de12677f0b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee63d4aa1eb84493a6973e76c453efe6', 'request_id': 'req-31ecef7e-50a6-4669-a94b-9de12677f0b9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.453007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-31ecef7e-50a6-4669-a94b-9de12677f0b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee63d4aa1eb84493a6973e76c453efe6', 'request_id': 'req-31ecef7e-50a6-4669-a94b-9de12677f0b9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.462468 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-31ecef7e-50a6-4669-a94b-9de12677f0b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee63d4aa1eb84493a6973e76c453efe6', 'request_id': 'req-31ecef7e-50a6-4669-a94b-9de12677f0b9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.489893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-31ecef7e-50a6-4669-a94b-9de12677f0b9 admin admin] The token cache is being invalidate because project ee63d4aa1eb84493a6973e76c453efe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:02.490738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-31ecef7e-50a6-4669-a94b-9de12677f0b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.491522 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-31ecef7e-50a6-4669-a94b-9de12677f0b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.493403 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1001/1985] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:02 2022] DELETE /identity/v3/projects/ee63d4aa1eb84493a6973e76c453efe6 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:02.500097 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb64e5-57ae-4703-a58f-0f32a536eeb5 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:02.500551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb64e5-57ae-4703-a58f-0f32a536eeb5 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:02.500672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb64e5-57ae-4703-a58f-0f32a536eeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:02.547952 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-14bb64e5-57ae-4703-a58f-0f32a536eeb5 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:02.549202 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-14bb64e5-57ae-4703-a58f-0f32a536eeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:02.616364 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 985/1986] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:02.625563 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047357dd-9a84-4365-99d3-f0982e559090 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:02.625854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047357dd-9a84-4365-99d3-f0982e559090 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:02.626135 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047357dd-9a84-4365-99d3-f0982e559090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:02.655195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-047357dd-9a84-4365-99d3-f0982e559090 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:02.655195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-047357dd-9a84-4365-99d3-f0982e559090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:02.658813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1002/1987] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:02.662680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb64e5-57ae-4703-a58f-0f32a536eeb5 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:02.681860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb64e5-57ae-4703-a58f-0f32a536eeb5 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:02.683036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e3b30-3a1a-4473-8ba3-6798f088a86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:02.684285 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e3b30-3a1a-4473-8ba3-6798f088a86f admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:02.684553 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e3b30-3a1a-4473-8ba3-6798f088a86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:02.685430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e3b30-3a1a-4473-8ba3-6798f088a86f admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:02.686282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e3b30-3a1a-4473-8ba3-6798f088a86f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:26:02.686472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e3b30-3a1a-4473-8ba3-6798f088a86f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:02.688295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e3b30-3a1a-4473-8ba3-6798f088a86f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:02.716246 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 986/1988] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:26:02.723818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047357dd-9a84-4365-99d3-f0982e559090 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:02.734857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047357dd-9a84-4365-99d3-f0982e559090 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:02.740941 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dddfbd-c8c6-4c93-b078-b298570924da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:02.741787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dddfbd-c8c6-4c93-b078-b298570924da admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:02.742134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dddfbd-c8c6-4c93-b078-b298570924da admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:02.742507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dddfbd-c8c6-4c93-b078-b298570924da admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:02.751825 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dddfbd-c8c6-4c93-b078-b298570924da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:02.751825 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dddfbd-c8c6-4c93-b078-b298570924da admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:02.836150 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1003/1989] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:02 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:02.840905 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e3b30-3a1a-4473-8ba3-6798f088a86f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:02.851423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e3b30-3a1a-4473-8ba3-6798f088a86f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:02.930361 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e7360-2e20-4dcc-b36e-15efa58a3d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:02.931075 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e7360-2e20-4dcc-b36e-15efa58a3d14 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:02.931387 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e7360-2e20-4dcc-b36e-15efa58a3d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:02.931672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e7360-2e20-4dcc-b36e-15efa58a3d14 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:02.932317 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e7360-2e20-4dcc-b36e-15efa58a3d14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:02.934325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e7360-2e20-4dcc-b36e-15efa58a3d14 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:02.989523 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:02.998020 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 987/1990] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:02 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:03.002010 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dddfbd-c8c6-4c93-b078-b298570924da admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:03.012700 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dddfbd-c8c6-4c93-b078-b298570924da admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:03.019420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea8cdf-4f28-4dc3-b9be-8b209fee63ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:03.020044 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea8cdf-4f28-4dc3-b9be-8b209fee63ff admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.020325 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea8cdf-4f28-4dc3-b9be-8b209fee63ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.020600 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea8cdf-4f28-4dc3-b9be-8b209fee63ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.022401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea8cdf-4f28-4dc3-b9be-8b209fee63ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:03.022868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea8cdf-4f28-4dc3-b9be-8b209fee63ff admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:03.040925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1004/1991] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:26:03.042116 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e7360-2e20-4dcc-b36e-15efa58a3d14 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:03.056609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e7360-2e20-4dcc-b36e-15efa58a3d14 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:03.062399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3aadd-5afc-4879-8e15-690718182c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:03.063062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3aadd-5afc-4879-8e15-690718182c37 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.063374 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3aadd-5afc-4879-8e15-690718182c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.067331 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3aadd-5afc-4879-8e15-690718182c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.067331 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3aadd-5afc-4879-8e15-690718182c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:03.067706 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3aadd-5afc-4879-8e15-690718182c37 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:03.079088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 988/1992] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:03.083466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea8cdf-4f28-4dc3-b9be-8b209fee63ff admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:03.094125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea8cdf-4f28-4dc3-b9be-8b209fee63ff admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:03.099087 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296e456-8f99-4be8-a8db-ed1af16c05da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:03.099805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296e456-8f99-4be8-a8db-ed1af16c05da admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.100087 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296e456-8f99-4be8-a8db-ed1af16c05da admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.100363 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296e456-8f99-4be8-a8db-ed1af16c05da admin admin] PATH_INFO: `/v3/projects/3c1d53a091884e578f12887f1357a3bd/users/72408b32d16f401292ca1506623989d5/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.120383 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296e456-8f99-4be8-a8db-ed1af16c05da admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c1d53a091884e578f12887f1357a3bd, user_id=72408b32d16f401292ca1506623989d5, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:03.124566 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296e456-8f99-4be8-a8db-ed1af16c05da admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:03.286850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1005/1993] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:03 2022] PUT /identity/v3/projects/3c1d53a091884e578f12887f1357a3bd/users/72408b32d16f401292ca1506623989d5/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:03.298185 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b0820-4f9d-45cc-af42-d0037b07c2c2 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.298518 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b0820-4f9d-45cc-af42-d0037b07c2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.298800 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b0820-4f9d-45cc-af42-d0037b07c2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.337718 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-8f9b0820-4f9d-45cc-af42-d0037b07c2c2 None None] MFA Rules not processed for user `72408b32d16f401292ca1506623989d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:03.339198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-8f9b0820-4f9d-45cc-af42-d0037b07c2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:03.442004 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 989/1994] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:03.453891 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296e456-8f99-4be8-a8db-ed1af16c05da admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:03.456418 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a296e456-8f99-4be8-a8db-ed1af16c05da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:03.571056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296e456-8f99-4be8-a8db-ed1af16c05da admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:03.575940 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a02ffc5-ab82-4deb-95c0-a4ecfbd0017d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:03.576789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02ffc5-ab82-4deb-95c0-a4ecfbd0017d service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.577073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02ffc5-ab82-4deb-95c0-a4ecfbd0017d service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.577348 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02ffc5-ab82-4deb-95c0-a4ecfbd0017d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.588038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a02ffc5-ab82-4deb-95c0-a4ecfbd0017d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:03.590252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a02ffc5-ab82-4deb-95c0-a4ecfbd0017d service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:03.603907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1006/1995] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:05.174585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b0820-4f9d-45cc-af42-d0037b07c2c2 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:05.181537 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-8f9b0820-4f9d-45cc-af42-d0037b07c2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:05.269572 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b0820-4f9d-45cc-af42-d0037b07c2c2 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:05.278960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256756df-7d39-4600-89fb-4b6fe8c95ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:05.279209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256756df-7d39-4600-89fb-4b6fe8c95ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:05.279498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256756df-7d39-4600-89fb-4b6fe8c95ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:05.279776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256756df-7d39-4600-89fb-4b6fe8c95ce0 admin admin] PATH_INFO: `/v3/users/facd975a155a4d3a975f3989b6c72f6d` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:05.280907 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256756df-7d39-4600-89fb-4b6fe8c95ce0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=facd975a155a4d3a975f3989b6c72f6d)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:05.282806 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256756df-7d39-4600-89fb-4b6fe8c95ce0 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:05.532861 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-256756df-7d39-4600-89fb-4b6fe8c95ce0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'facd975a155a4d3a975f3989b6c72f6d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:05.548029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-256756df-7d39-4600-89fb-4b6fe8c95ce0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'facd975a155a4d3a975f3989b6c72f6d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:05.556070 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-256756df-7d39-4600-89fb-4b6fe8c95ce0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'facd975a155a4d3a975f3989b6c72f6d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:05.560681 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:05.611476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-256756df-7d39-4600-89fb-4b6fe8c95ce0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'facd975a155a4d3a975f3989b6c72f6d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:05.612080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-256756df-7d39-4600-89fb-4b6fe8c95ce0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'facd975a155a4d3a975f3989b6c72f6d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:05.619055 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 990/1996] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:05 2022] DELETE /identity/v3/users/facd975a155a4d3a975f3989b6c72f6d => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:05.624354 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eba080-bed4-40d4-af38-b8983e781f43 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:05.624906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eba080-bed4-40d4-af38-b8983e781f43 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:05.624906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eba080-bed4-40d4-af38-b8983e781f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:05.645185 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f3eba080-bed4-40d4-af38-b8983e781f43 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:05.646437 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f3eba080-bed4-40d4-af38-b8983e781f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:05.669128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:05.733672 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1007/1997] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:05.745301 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256756df-7d39-4600-89fb-4b6fe8c95ce0 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:05.748932 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-256756df-7d39-4600-89fb-4b6fe8c95ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:05.822077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256756df-7d39-4600-89fb-4b6fe8c95ce0 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:05.828224 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78af1a-735a-46ed-8167-434fb5f25e10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:05.828842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78af1a-735a-46ed-8167-434fb5f25e10 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:05.829141 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78af1a-735a-46ed-8167-434fb5f25e10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:05.829413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78af1a-735a-46ed-8167-434fb5f25e10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:05.839457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78af1a-735a-46ed-8167-434fb5f25e10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:05.841219 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78af1a-735a-46ed-8167-434fb5f25e10 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:05.852575 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 991/1998] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:06.312439 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eba080-bed4-40d4-af38-b8983e781f43 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:06.313984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f3eba080-bed4-40d4-af38-b8983e781f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:06.352896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eba080-bed4-40d4-af38-b8983e781f43 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:06.352896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af8c77-6755-4197-8c30-91fc9169abe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:06.352896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af8c77-6755-4197-8c30-91fc9169abe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:06.352896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af8c77-6755-4197-8c30-91fc9169abe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:06.353641 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af8c77-6755-4197-8c30-91fc9169abe0 admin admin] PATH_INFO: `/v3/projects/1f71d072c0244f5785d2830d4f4a7ff2` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:06.353641 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af8c77-6755-4197-8c30-91fc9169abe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f71d072c0244f5785d2830d4f4a7ff2)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:06.355089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af8c77-6755-4197-8c30-91fc9169abe0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:06.470652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e8af8c77-6755-4197-8c30-91fc9169abe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f71d072c0244f5785d2830d4f4a7ff2', 'request_id': 'req-e8af8c77-6755-4197-8c30-91fc9169abe0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:06.594089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e8af8c77-6755-4197-8c30-91fc9169abe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f71d072c0244f5785d2830d4f4a7ff2', 'request_id': 'req-e8af8c77-6755-4197-8c30-91fc9169abe0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:06.601273 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e8af8c77-6755-4197-8c30-91fc9169abe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f71d072c0244f5785d2830d4f4a7ff2', 'request_id': 'req-e8af8c77-6755-4197-8c30-91fc9169abe0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:06.612802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e8af8c77-6755-4197-8c30-91fc9169abe0 admin admin] The token cache is being invalidate because project 1f71d072c0244f5785d2830d4f4a7ff2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:06.613121 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e8af8c77-6755-4197-8c30-91fc9169abe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:06.614896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e8af8c77-6755-4197-8c30-91fc9169abe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:06.615475 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1008/1999] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:06 2022] DELETE /identity/v3/projects/1f71d072c0244f5785d2830d4f4a7ff2 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:06.619422 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78af1a-735a-46ed-8167-434fb5f25e10 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:06.633068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78af1a-735a-46ed-8167-434fb5f25e10 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:06.694111 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d03334-a985-48b0-8f75-c057650e0c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:06.694782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d03334-a985-48b0-8f75-c057650e0c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:06.695216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d03334-a985-48b0-8f75-c057650e0c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:06.695513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d03334-a985-48b0-8f75-c057650e0c71 admin admin] PATH_INFO: `/v3/users/25d41f7631ad49c28f080165ba9dfbcf` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:06.696736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d03334-a985-48b0-8f75-c057650e0c71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d41f7631ad49c28f080165ba9dfbcf)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:06.698673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d03334-a985-48b0-8f75-c057650e0c71 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:06.917399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-32d03334-a985-48b0-8f75-c057650e0c71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d41f7631ad49c28f080165ba9dfbcf'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:06.929597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-32d03334-a985-48b0-8f75-c057650e0c71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d41f7631ad49c28f080165ba9dfbcf'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:06.936496 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-32d03334-a985-48b0-8f75-c057650e0c71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d41f7631ad49c28f080165ba9dfbcf'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:07.022670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-32d03334-a985-48b0-8f75-c057650e0c71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d41f7631ad49c28f080165ba9dfbcf'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:07.023459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-32d03334-a985-48b0-8f75-c057650e0c71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d41f7631ad49c28f080165ba9dfbcf'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:07.029919 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 992/2000] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:06 2022] DELETE /identity/v3/users/25d41f7631ad49c28f080165ba9dfbcf => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:07.516714 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af8c77-6755-4197-8c30-91fc9169abe0 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:07.518748 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e8af8c77-6755-4197-8c30-91fc9169abe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:07.534662 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af8c77-6755-4197-8c30-91fc9169abe0 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:07.601899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8df024-7da4-4363-af3e-23e5bda12ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:07.602654 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8df024-7da4-4363-af3e-23e5bda12ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:07.602999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8df024-7da4-4363-af3e-23e5bda12ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:07.603474 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8df024-7da4-4363-af3e-23e5bda12ed2 admin admin] PATH_INFO: `/v3/projects/ad42fe1d969342ce954faca5dd671db4` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:07.605512 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8df024-7da4-4363-af3e-23e5bda12ed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad42fe1d969342ce954faca5dd671db4)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:07.606843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8df024-7da4-4363-af3e-23e5bda12ed2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:07.736158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:07.737494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4d8df024-7da4-4363-af3e-23e5bda12ed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad42fe1d969342ce954faca5dd671db4', 'request_id': 'req-4d8df024-7da4-4363-af3e-23e5bda12ed2'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:07.822455 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4d8df024-7da4-4363-af3e-23e5bda12ed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad42fe1d969342ce954faca5dd671db4', 'request_id': 'req-4d8df024-7da4-4363-af3e-23e5bda12ed2'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:07.829889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4d8df024-7da4-4363-af3e-23e5bda12ed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad42fe1d969342ce954faca5dd671db4', 'request_id': 'req-4d8df024-7da4-4363-af3e-23e5bda12ed2'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:07.840468 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4d8df024-7da4-4363-af3e-23e5bda12ed2 admin admin] The token cache is being invalidate because project ad42fe1d969342ce954faca5dd671db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:07.840757 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4d8df024-7da4-4363-af3e-23e5bda12ed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:07.841393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4d8df024-7da4-4363-af3e-23e5bda12ed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:07.842879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1009/2001] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:07 2022] DELETE /identity/v3/projects/ad42fe1d969342ce954faca5dd671db4 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:07.847977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a09d5e-5cf0-418d-a691-89aba3b0da52 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:07.848444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a09d5e-5cf0-418d-a691-89aba3b0da52 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:07.848532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a09d5e-5cf0-418d-a691-89aba3b0da52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:07.868215 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-66a09d5e-5cf0-418d-a691-89aba3b0da52 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:07.869434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-66a09d5e-5cf0-418d-a691-89aba3b0da52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:07.882120 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:07.928849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 993/2002] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:07.937405 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbf0e2-2730-47c2-aeeb-c796957058f7 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:07.937691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbf0e2-2730-47c2-aeeb-c796957058f7 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:07.937972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbf0e2-2730-47c2-aeeb-c796957058f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:07.957945 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-3dfbf0e2-2730-47c2-aeeb-c796957058f7 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:07.959161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3dfbf0e2-2730-47c2-aeeb-c796957058f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:07.967452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1010/2003] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:07.971842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a09d5e-5cf0-418d-a691-89aba3b0da52 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:07.981514 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a09d5e-5cf0-418d-a691-89aba3b0da52 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:07.986110 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f396cc-d84d-4fcd-a525-ff08b320fce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:07.986734 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f396cc-d84d-4fcd-a525-ff08b320fce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:07.987009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f396cc-d84d-4fcd-a525-ff08b320fce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:07.987300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f396cc-d84d-4fcd-a525-ff08b320fce1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:07.987945 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f396cc-d84d-4fcd-a525-ff08b320fce1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:26:07.988263 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f396cc-d84d-4fcd-a525-ff08b320fce1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:07.989854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f396cc-d84d-4fcd-a525-ff08b320fce1 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:08.002067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 994/2004] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:26:08.006601 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbf0e2-2730-47c2-aeeb-c796957058f7 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:08.016550 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbf0e2-2730-47c2-aeeb-c796957058f7 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:08.021292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e35b31-c6cf-49d0-9169-c370a8974067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:08.021892 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e35b31-c6cf-49d0-9169-c370a8974067 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:08.022176 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e35b31-c6cf-49d0-9169-c370a8974067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:08.022445 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e35b31-c6cf-49d0-9169-c370a8974067 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:08.023093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e35b31-c6cf-49d0-9169-c370a8974067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:08.025027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e35b31-c6cf-49d0-9169-c370a8974067 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:08.110495 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1011/2005] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:08 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:08.114971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f396cc-d84d-4fcd-a525-ff08b320fce1 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:08.125791 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f396cc-d84d-4fcd-a525-ff08b320fce1 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:08.182601 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a4abf-acf7-42fc-a1de-8979f6eb0aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:08.183125 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a4abf-acf7-42fc-a1de-8979f6eb0aed admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:08.183983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a4abf-acf7-42fc-a1de-8979f6eb0aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:08.184260 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a4abf-acf7-42fc-a1de-8979f6eb0aed admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:08.184897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a4abf-acf7-42fc-a1de-8979f6eb0aed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:08.186939 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a4abf-acf7-42fc-a1de-8979f6eb0aed admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:08.274793 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 995/2006] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:08 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:08.279225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e35b31-c6cf-49d0-9169-c370a8974067 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:08.290849 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e35b31-c6cf-49d0-9169-c370a8974067 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:08.307649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927bf25-b96d-48bf-8342-4ba2f42f8d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:08.307649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927bf25-b96d-48bf-8342-4ba2f42f8d9b admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:08.307649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927bf25-b96d-48bf-8342-4ba2f42f8d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:08.307649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927bf25-b96d-48bf-8342-4ba2f42f8d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:08.307977 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927bf25-b96d-48bf-8342-4ba2f42f8d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:08.307977 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927bf25-b96d-48bf-8342-4ba2f42f8d9b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:08.314155 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1012/2007] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:08.315591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a4abf-acf7-42fc-a1de-8979f6eb0aed admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:08.326313 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a4abf-acf7-42fc-a1de-8979f6eb0aed admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:08.331392 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4110cd-b63f-4820-8d8c-9143068d65e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:08.332182 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4110cd-b63f-4820-8d8c-9143068d65e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:08.332361 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4110cd-b63f-4820-8d8c-9143068d65e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:08.332704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4110cd-b63f-4820-8d8c-9143068d65e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:08.333370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4110cd-b63f-4820-8d8c-9143068d65e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:08.334935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4110cd-b63f-4820-8d8c-9143068d65e8 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:08.358860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 996/2008] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:08.363193 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927bf25-b96d-48bf-8342-4ba2f42f8d9b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:08.378156 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927bf25-b96d-48bf-8342-4ba2f42f8d9b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:08.383483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12f905-4836-48db-93a4-899b977b0e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:08.384185 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12f905-4836-48db-93a4-899b977b0e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:08.384510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12f905-4836-48db-93a4-899b977b0e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:08.384796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12f905-4836-48db-93a4-899b977b0e2a admin admin] PATH_INFO: `/v3/projects/47fc574b240e4a09bb83d1d2f20558c4/users/e81d8e2a42f34cc0b4f3ca54eb203f59/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:08.399648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12f905-4836-48db-93a4-899b977b0e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=47fc574b240e4a09bb83d1d2f20558c4, user_id=e81d8e2a42f34cc0b4f3ca54eb203f59, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:08.402311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12f905-4836-48db-93a4-899b977b0e2a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:08.497314 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1013/2009] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:08 2022] PUT /identity/v3/projects/47fc574b240e4a09bb83d1d2f20558c4/users/e81d8e2a42f34cc0b4f3ca54eb203f59/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:26:08.502363 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713ea1d-ae1b-42c4-8c09-6ec6f8765cc0 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:08.502647 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713ea1d-ae1b-42c4-8c09-6ec6f8765cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:08.502923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713ea1d-ae1b-42c4-8c09-6ec6f8765cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:08.521846 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-6713ea1d-ae1b-42c4-8c09-6ec6f8765cc0 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:08.523056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-6713ea1d-ae1b-42c4-8c09-6ec6f8765cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:08.581056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 997/2010] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:08.591601 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12f905-4836-48db-93a4-899b977b0e2a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:08.593385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ad12f905-4836-48db-93a4-899b977b0e2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:08.650507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12f905-4836-48db-93a4-899b977b0e2a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:08.654850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67656988-d650-42da-8c4e-70cd51afe4dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:08.655505 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67656988-d650-42da-8c4e-70cd51afe4dd service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:08.655786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67656988-d650-42da-8c4e-70cd51afe4dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:08.656063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67656988-d650-42da-8c4e-70cd51afe4dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:08.665019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67656988-d650-42da-8c4e-70cd51afe4dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:08.666439 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67656988-d650-42da-8c4e-70cd51afe4dd service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:08.676473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1014/2011] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:10.011571 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4bc70-5387-417c-9e4c-5f3274dc100c None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:10.012113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4bc70-5387-417c-9e4c-5f3274dc100c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:10.012113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4bc70-5387-417c-9e4c-5f3274dc100c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:10.050893 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-68e4bc70-5387-417c-9e4c-5f3274dc100c None None] MFA Rules not processed for user `e81d8e2a42f34cc0b4f3ca54eb203f59`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:10.052101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-68e4bc70-5387-417c-9e4c-5f3274dc100c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:10.066576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:10.153857 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 998/2012] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:10.372584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67656988-d650-42da-8c4e-70cd51afe4dd service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:10.374122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-67656988-d650-42da-8c4e-70cd51afe4dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:10.432612 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67656988-d650-42da-8c4e-70cd51afe4dd service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:10.436346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431a050-62f0-4566-ac91-a604b0216281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:10.436955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431a050-62f0-4566-ac91-a604b0216281 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:10.437239 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431a050-62f0-4566-ac91-a604b0216281 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:10.437519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431a050-62f0-4566-ac91-a604b0216281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:10.446057 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431a050-62f0-4566-ac91-a604b0216281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:10.447528 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431a050-62f0-4566-ac91-a604b0216281 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:10.457229 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1015/2013] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:13.841465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880b08d-2cd8-46ae-a8af-759925893811 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:13.841956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880b08d-2cd8-46ae-a8af-759925893811 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:13.841956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880b08d-2cd8-46ae-a8af-759925893811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:13.868218 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-5880b08d-2cd8-46ae-a8af-759925893811 None None] MFA Rules not processed for user `e81d8e2a42f34cc0b4f3ca54eb203f59`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:13.869467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5880b08d-2cd8-46ae-a8af-759925893811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:13.877417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 999/2014] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:17.390354 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431a050-62f0-4566-ac91-a604b0216281 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:17.392606 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2431a050-62f0-4566-ac91-a604b0216281 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:17.408731 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431a050-62f0-4566-ac91-a604b0216281 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:17.414020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2acb0c-7d04-4c62-b1ca-9998c7ed11b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:17.415686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2acb0c-7d04-4c62-b1ca-9998c7ed11b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:17.416191 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2acb0c-7d04-4c62-b1ca-9998c7ed11b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:17.416637 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2acb0c-7d04-4c62-b1ca-9998c7ed11b9 admin admin] PATH_INFO: `/v3/users/c84f30af07de4ba1a99f03cfe56cad45` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:17.419123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2acb0c-7d04-4c62-b1ca-9998c7ed11b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c84f30af07de4ba1a99f03cfe56cad45)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:17.421522 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2acb0c-7d04-4c62-b1ca-9998c7ed11b9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:17.446610 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:17.669454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7f2acb0c-7d04-4c62-b1ca-9998c7ed11b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c84f30af07de4ba1a99f03cfe56cad45'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:17.682038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7f2acb0c-7d04-4c62-b1ca-9998c7ed11b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c84f30af07de4ba1a99f03cfe56cad45'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:17.694590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7f2acb0c-7d04-4c62-b1ca-9998c7ed11b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c84f30af07de4ba1a99f03cfe56cad45'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:17.833563 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7f2acb0c-7d04-4c62-b1ca-9998c7ed11b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c84f30af07de4ba1a99f03cfe56cad45'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:17.834475 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7f2acb0c-7d04-4c62-b1ca-9998c7ed11b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c84f30af07de4ba1a99f03cfe56cad45'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:17.843420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1016/2015] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:17 2022] DELETE /identity/v3/users/c84f30af07de4ba1a99f03cfe56cad45 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:17.847935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c62f7-1001-4e80-bfb7-ac151555a723 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:17.848447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c62f7-1001-4e80-bfb7-ac151555a723 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:17.848604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c62f7-1001-4e80-bfb7-ac151555a723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:17.872693 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-871c62f7-1001-4e80-bfb7-ac151555a723 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:17.872693 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-871c62f7-1001-4e80-bfb7-ac151555a723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:17.976179 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1000/2016] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:17.977964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2acb0c-7d04-4c62-b1ca-9998c7ed11b9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:17.979786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7f2acb0c-7d04-4c62-b1ca-9998c7ed11b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:18.045556 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2acb0c-7d04-4c62-b1ca-9998c7ed11b9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:18.049336 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2cea8-00ef-40ba-bbd9-434274563c6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:18.049977 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2cea8-00ef-40ba-bbd9-434274563c6c service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:18.050263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2cea8-00ef-40ba-bbd9-434274563c6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:18.050542 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2cea8-00ef-40ba-bbd9-434274563c6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:18.061095 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2cea8-00ef-40ba-bbd9-434274563c6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:18.063029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2cea8-00ef-40ba-bbd9-434274563c6c service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:18.076389 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1017/2017] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:26:18.743355 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c62f7-1001-4e80-bfb7-ac151555a723 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:18.744921 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-871c62f7-1001-4e80-bfb7-ac151555a723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:18.758781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c62f7-1001-4e80-bfb7-ac151555a723 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:18.764656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1f369-a7fa-45d9-be23-9863a41a32fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:18.765614 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1f369-a7fa-45d9-be23-9863a41a32fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:18.766291 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1f369-a7fa-45d9-be23-9863a41a32fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:18.766662 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1f369-a7fa-45d9-be23-9863a41a32fa admin admin] PATH_INFO: `/v3/projects/c3d6c73709914b8286ae13a68710fce3` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:18.770039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1f369-a7fa-45d9-be23-9863a41a32fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d6c73709914b8286ae13a68710fce3)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:18.770153 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1f369-a7fa-45d9-be23-9863a41a32fa admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:18.873000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2cea8-00ef-40ba-bbd9-434274563c6c service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:18.875039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ffe2cea8-00ef-40ba-bbd9-434274563c6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:18.888037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2cea8-00ef-40ba-bbd9-434274563c6c service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:18.909803 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-2ee1f369-a7fa-45d9-be23-9863a41a32fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d6c73709914b8286ae13a68710fce3', 'request_id': 'req-2ee1f369-a7fa-45d9-be23-9863a41a32fa'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:18.912631 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526d50f-5057-4cc7-9e01-c06c450aeb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:18.913296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526d50f-5057-4cc7-9e01-c06c450aeb2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:18.913587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526d50f-5057-4cc7-9e01-c06c450aeb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:18.913864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526d50f-5057-4cc7-9e01-c06c450aeb2c admin admin] PATH_INFO: `/v3/users/972c5b4258504adc8bca9cccc0376f91` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:18.915760 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526d50f-5057-4cc7-9e01-c06c450aeb2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=972c5b4258504adc8bca9cccc0376f91)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:18.918281 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526d50f-5057-4cc7-9e01-c06c450aeb2c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:18.920890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-2ee1f369-a7fa-45d9-be23-9863a41a32fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d6c73709914b8286ae13a68710fce3', 'request_id': 'req-2ee1f369-a7fa-45d9-be23-9863a41a32fa'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:18.982359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-2ee1f369-a7fa-45d9-be23-9863a41a32fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d6c73709914b8286ae13a68710fce3', 'request_id': 'req-2ee1f369-a7fa-45d9-be23-9863a41a32fa'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:18.995310 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-2ee1f369-a7fa-45d9-be23-9863a41a32fa admin admin] The token cache is being invalidate because project c3d6c73709914b8286ae13a68710fce3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:18.995628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-2ee1f369-a7fa-45d9-be23-9863a41a32fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:18.996283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-2ee1f369-a7fa-45d9-be23-9863a41a32fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:18.997857 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1001/2018] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:18 2022] DELETE /identity/v3/projects/c3d6c73709914b8286ae13a68710fce3 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:19.002788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff74f7-cb2a-4ff4-9549-c80da73c6653 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:19.005398 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff74f7-cb2a-4ff4-9549-c80da73c6653 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:19.005398 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff74f7-cb2a-4ff4-9549-c80da73c6653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:19.023495 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-1cff74f7-cb2a-4ff4-9549-c80da73c6653 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:19.024733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1cff74f7-cb2a-4ff4-9549-c80da73c6653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:19.053802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:19.072290 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4526d50f-5057-4cc7-9e01-c06c450aeb2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '972c5b4258504adc8bca9cccc0376f91'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:19.090070 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4526d50f-5057-4cc7-9e01-c06c450aeb2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '972c5b4258504adc8bca9cccc0376f91'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:19.097986 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4526d50f-5057-4cc7-9e01-c06c450aeb2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '972c5b4258504adc8bca9cccc0376f91'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:19.121488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1002/2019] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:19.129681 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c8e5e-635e-43c3-b6c6-1f4fdebe84ab None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:19.130041 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c8e5e-635e-43c3-b6c6-1f4fdebe84ab None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:19.130391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c8e5e-635e-43c3-b6c6-1f4fdebe84ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:19.158315 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-7a5c8e5e-635e-43c3-b6c6-1f4fdebe84ab None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:19.161602 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7a5c8e5e-635e-43c3-b6c6-1f4fdebe84ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:19.165483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4526d50f-5057-4cc7-9e01-c06c450aeb2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '972c5b4258504adc8bca9cccc0376f91'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:19.166222 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4526d50f-5057-4cc7-9e01-c06c450aeb2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '972c5b4258504adc8bca9cccc0376f91'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:19.177224 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1018/2020] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:18 2022] DELETE /identity/v3/users/972c5b4258504adc8bca9cccc0376f91 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:19.183252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fbb4b-748d-484b-8a14-c2d88056a823 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:19.183755 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fbb4b-748d-484b-8a14-c2d88056a823 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:19.184249 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fbb4b-748d-484b-8a14-c2d88056a823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:19.187710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1003/2021] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:19.192584 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c8e5e-635e-43c3-b6c6-1f4fdebe84ab None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:19.195580 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:19.204012 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c8e5e-635e-43c3-b6c6-1f4fdebe84ab None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:19.209541 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-443fbb4b-748d-484b-8a14-c2d88056a823 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:19.211491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-443fbb4b-748d-484b-8a14-c2d88056a823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:19.311887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1019/2022] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:19.323518 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443fbb4b-748d-484b-8a14-c2d88056a823 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:19.325694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-443fbb4b-748d-484b-8a14-c2d88056a823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:19.336423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70be6fe-7565-4e3c-8f9d-359d8ff055e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:19.337050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70be6fe-7565-4e3c-8f9d-359d8ff055e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:19.337330 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70be6fe-7565-4e3c-8f9d-359d8ff055e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:19.337612 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70be6fe-7565-4e3c-8f9d-359d8ff055e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:19.338308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70be6fe-7565-4e3c-8f9d-359d8ff055e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:26:19.338613 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70be6fe-7565-4e3c-8f9d-359d8ff055e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:19.341597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70be6fe-7565-4e3c-8f9d-359d8ff055e6 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:19.363257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1004/2023] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:26:19.376685 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70be6fe-7565-4e3c-8f9d-359d8ff055e6 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:19.392197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70be6fe-7565-4e3c-8f9d-359d8ff055e6 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:19.403561 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266a01c-3989-4b23-a726-5a8c12f05eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:19.404623 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266a01c-3989-4b23-a726-5a8c12f05eed admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:19.405054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266a01c-3989-4b23-a726-5a8c12f05eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:19.405475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266a01c-3989-4b23-a726-5a8c12f05eed admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:19.406663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266a01c-3989-4b23-a726-5a8c12f05eed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:19.408898 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266a01c-3989-4b23-a726-5a8c12f05eed admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:19.424513 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443fbb4b-748d-484b-8a14-c2d88056a823 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:19.429993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4a831-e6a5-40d0-829c-8a0ff1bf5f8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:19.430638 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4a831-e6a5-40d0-829c-8a0ff1bf5f8a service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:19.430922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4a831-e6a5-40d0-829c-8a0ff1bf5f8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:19.431217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4a831-e6a5-40d0-829c-8a0ff1bf5f8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:19.443073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4a831-e6a5-40d0-829c-8a0ff1bf5f8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:19.444476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4a831-e6a5-40d0-829c-8a0ff1bf5f8a service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:19.455763 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1020/2024] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:26:19.487204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1005/2025] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:19 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:19.491462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4a831-e6a5-40d0-829c-8a0ff1bf5f8a service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:19.505530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4a831-e6a5-40d0-829c-8a0ff1bf5f8a service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:19.570530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5d429-f4d5-443e-bd3a-5b3e8a80b54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:19.571153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5d429-f4d5-443e-bd3a-5b3e8a80b54b admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:19.571459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5d429-f4d5-443e-bd3a-5b3e8a80b54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:19.571731 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5d429-f4d5-443e-bd3a-5b3e8a80b54b admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:19.572375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e5d429-f4d5-443e-bd3a-5b3e8a80b54b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:19.574345 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e5d429-f4d5-443e-bd3a-5b3e8a80b54b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:19.689971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1021/2026] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:19 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:19.690429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266a01c-3989-4b23-a726-5a8c12f05eed admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:19.696538 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266a01c-3989-4b23-a726-5a8c12f05eed admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:19.701952 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f31708-31dc-4d27-acdb-6e644390a11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:19.702664 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f31708-31dc-4d27-acdb-6e644390a11a admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:19.703007 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f31708-31dc-4d27-acdb-6e644390a11a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:19.703384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f31708-31dc-4d27-acdb-6e644390a11a admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:19.704494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f31708-31dc-4d27-acdb-6e644390a11a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:19.705805 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f31708-31dc-4d27-acdb-6e644390a11a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:19.720122 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1006/2027] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:19.721399 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5d429-f4d5-443e-bd3a-5b3e8a80b54b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:19.733126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5d429-f4d5-443e-bd3a-5b3e8a80b54b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:19.738730 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269d8a2-2f3c-430e-98fd-433b2c6b8026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:19.739468 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269d8a2-2f3c-430e-98fd-433b2c6b8026 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:19.746833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269d8a2-2f3c-430e-98fd-433b2c6b8026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:19.746833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269d8a2-2f3c-430e-98fd-433b2c6b8026 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:19.746833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269d8a2-2f3c-430e-98fd-433b2c6b8026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:19.746833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269d8a2-2f3c-430e-98fd-433b2c6b8026 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:19.752986 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1022/2028] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:19.757202 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f31708-31dc-4d27-acdb-6e644390a11a admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:19.768307 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f31708-31dc-4d27-acdb-6e644390a11a admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:19.773061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a29066-afd4-4772-96ac-a89b0f99c45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:19.773765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a29066-afd4-4772-96ac-a89b0f99c45f admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:19.775545 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a29066-afd4-4772-96ac-a89b0f99c45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:19.775545 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a29066-afd4-4772-96ac-a89b0f99c45f admin admin] PATH_INFO: `/v3/projects/bceff2dbe9f742508a2b57e512dc85c1/users/f0ba1bdc42bc4595931da1732364e8be/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:19.790650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a29066-afd4-4772-96ac-a89b0f99c45f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bceff2dbe9f742508a2b57e512dc85c1, user_id=f0ba1bdc42bc4595931da1732364e8be, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:19.793755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a29066-afd4-4772-96ac-a89b0f99c45f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:19.885372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1007/2029] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:19 2022] PUT /identity/v3/projects/bceff2dbe9f742508a2b57e512dc85c1/users/f0ba1bdc42bc4595931da1732364e8be/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:26:19.904881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4d7ab-6916-42c1-9dcf-1cb34dd23257 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:19.904881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4d7ab-6916-42c1-9dcf-1cb34dd23257 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:19.904881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4d7ab-6916-42c1-9dcf-1cb34dd23257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:19.930493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-51d4d7ab-6916-42c1-9dcf-1cb34dd23257 None None] MFA Rules not processed for user `f0ba1bdc42bc4595931da1732364e8be`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:19.930493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-51d4d7ab-6916-42c1-9dcf-1cb34dd23257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:19.932805 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a29066-afd4-4772-96ac-a89b0f99c45f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:19.940856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-24a29066-afd4-4772-96ac-a89b0f99c45f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:20.008131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1023/2030] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:20.017518 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a29066-afd4-4772-96ac-a89b0f99c45f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:20.019373 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4d7ab-6916-42c1-9dcf-1cb34dd23257 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:20.025431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-51d4d7ab-6916-42c1-9dcf-1cb34dd23257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:20.025551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae0ff1-4bd1-44c5-9a91-81a74811356e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:20.025551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae0ff1-4bd1-44c5-9a91-81a74811356e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:20.025551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae0ff1-4bd1-44c5-9a91-81a74811356e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:20.025551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae0ff1-4bd1-44c5-9a91-81a74811356e admin admin] PATH_INFO: `/v3/projects/ff457732c9fc42ef9c7da7b2c15155f7` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:20.025906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae0ff1-4bd1-44c5-9a91-81a74811356e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff457732c9fc42ef9c7da7b2c15155f7)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:20.027355 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae0ff1-4bd1-44c5-9a91-81a74811356e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:20.045389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:20.084518 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4d7ab-6916-42c1-9dcf-1cb34dd23257 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:20.088169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34aacf-b9aa-4bdb-8067-db7c6b5d86d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:20.088791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34aacf-b9aa-4bdb-8067-db7c6b5d86d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:20.090017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34aacf-b9aa-4bdb-8067-db7c6b5d86d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:20.090304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34aacf-b9aa-4bdb-8067-db7c6b5d86d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:20.099043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34aacf-b9aa-4bdb-8067-db7c6b5d86d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:20.100499 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34aacf-b9aa-4bdb-8067-db7c6b5d86d2 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:20.109497 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1024/2031] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:26:20.134965 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34aacf-b9aa-4bdb-8067-db7c6b5d86d2 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:20.146240 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34aacf-b9aa-4bdb-8067-db7c6b5d86d2 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:20.150845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cca3a-55ca-4359-8d5d-29e346a1822d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:20.151490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cca3a-55ca-4359-8d5d-29e346a1822d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:20.152426 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cca3a-55ca-4359-8d5d-29e346a1822d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:20.152708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cca3a-55ca-4359-8d5d-29e346a1822d admin admin] PATH_INFO: `/v3/users/f0ba1bdc42bc4595931da1732364e8be` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:20.153848 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cca3a-55ca-4359-8d5d-29e346a1822d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0ba1bdc42bc4595931da1732364e8be)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:20.155799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cca3a-55ca-4359-8d5d-29e346a1822d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:20.194903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:20.200542 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-eeae0ff1-4bd1-44c5-9a91-81a74811356e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff457732c9fc42ef9c7da7b2c15155f7', 'request_id': 'req-eeae0ff1-4bd1-44c5-9a91-81a74811356e'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:20.207647 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-eeae0ff1-4bd1-44c5-9a91-81a74811356e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff457732c9fc42ef9c7da7b2c15155f7', 'request_id': 'req-eeae0ff1-4bd1-44c5-9a91-81a74811356e'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:20.290841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-eeae0ff1-4bd1-44c5-9a91-81a74811356e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff457732c9fc42ef9c7da7b2c15155f7', 'request_id': 'req-eeae0ff1-4bd1-44c5-9a91-81a74811356e'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:20.304175 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-eeae0ff1-4bd1-44c5-9a91-81a74811356e admin admin] The token cache is being invalidate because project ff457732c9fc42ef9c7da7b2c15155f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:20.304490 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-eeae0ff1-4bd1-44c5-9a91-81a74811356e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:20.305152 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-eeae0ff1-4bd1-44c5-9a91-81a74811356e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:20.306681 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1008/2032] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:19 2022] DELETE /identity/v3/projects/ff457732c9fc42ef9c7da7b2c15155f7 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:20.310926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae0ff1-4bd1-44c5-9a91-81a74811356e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:20.312296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-eeae0ff1-4bd1-44c5-9a91-81a74811356e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:20.356328 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2e2cca3a-55ca-4359-8d5d-29e346a1822d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0ba1bdc42bc4595931da1732364e8be'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:20.372666 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2e2cca3a-55ca-4359-8d5d-29e346a1822d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0ba1bdc42bc4595931da1732364e8be'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:20.379936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2e2cca3a-55ca-4359-8d5d-29e346a1822d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0ba1bdc42bc4595931da1732364e8be'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:20.380082 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae0ff1-4bd1-44c5-9a91-81a74811356e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:20.386894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bde0c-6973-4ad9-ac78-958c7b653ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:20.388088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bde0c-6973-4ad9-ac78-958c7b653ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:20.388430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bde0c-6973-4ad9-ac78-958c7b653ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:20.388808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bde0c-6973-4ad9-ac78-958c7b653ab4 admin admin] PATH_INFO: `/v3/users/75478d59dc58409599fd2da9ae0d55ff` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:20.390477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049bde0c-6973-4ad9-ac78-958c7b653ab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75478d59dc58409599fd2da9ae0d55ff)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:20.392800 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049bde0c-6973-4ad9-ac78-958c7b653ab4 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:20.633283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2e2cca3a-55ca-4359-8d5d-29e346a1822d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0ba1bdc42bc4595931da1732364e8be'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:20.633925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2e2cca3a-55ca-4359-8d5d-29e346a1822d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0ba1bdc42bc4595931da1732364e8be'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:20.640241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1025/2033] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:20 2022] DELETE /identity/v3/users/f0ba1bdc42bc4595931da1732364e8be => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:26:20.644636 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3823-284b-4247-9ca3-6c8752350a3e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:20.644932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3823-284b-4247-9ca3-6c8752350a3e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:20.645204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3823-284b-4247-9ca3-6c8752350a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:20.664756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-1aab3823-284b-4247-9ca3-6c8752350a3e None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:20.665978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1aab3823-284b-4247-9ca3-6c8752350a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:20.725432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1026/2034] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:20.735452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab3823-284b-4247-9ca3-6c8752350a3e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:20.736879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1aab3823-284b-4247-9ca3-6c8752350a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:20.792902 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-049bde0c-6973-4ad9-ac78-958c7b653ab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75478d59dc58409599fd2da9ae0d55ff'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:20.805291 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-049bde0c-6973-4ad9-ac78-958c7b653ab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75478d59dc58409599fd2da9ae0d55ff'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:20.805681 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab3823-284b-4247-9ca3-6c8752350a3e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:20.810748 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09f678-7400-422f-ab71-a84828cfe2c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:20.811523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09f678-7400-422f-ab71-a84828cfe2c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:20.811783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09f678-7400-422f-ab71-a84828cfe2c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:20.812061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09f678-7400-422f-ab71-a84828cfe2c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:20.814186 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-049bde0c-6973-4ad9-ac78-958c7b653ab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75478d59dc58409599fd2da9ae0d55ff'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:20.823375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09f678-7400-422f-ab71-a84828cfe2c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:20.825503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09f678-7400-422f-ab71-a84828cfe2c4 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:20.835604 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1027/2035] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:26:20.873489 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-049bde0c-6973-4ad9-ac78-958c7b653ab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75478d59dc58409599fd2da9ae0d55ff'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:20.874178 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-049bde0c-6973-4ad9-ac78-958c7b653ab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75478d59dc58409599fd2da9ae0d55ff'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:20.881389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1009/2036] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:20 2022] DELETE /identity/v3/users/75478d59dc58409599fd2da9ae0d55ff => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:26:21.108304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:21.395000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09f678-7400-422f-ab71-a84828cfe2c4 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:21.396565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-aa09f678-7400-422f-ab71-a84828cfe2c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:21.413147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09f678-7400-422f-ab71-a84828cfe2c4 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:21.419438 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bde0c-6973-4ad9-ac78-958c7b653ab4 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:21.422198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-049bde0c-6973-4ad9-ac78-958c7b653ab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:21.440776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:21.497564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65e269-817d-4a88-a6b5-37fb8919f050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:21.498233 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65e269-817d-4a88-a6b5-37fb8919f050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:21.498522 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65e269-817d-4a88-a6b5-37fb8919f050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:21.498795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65e269-817d-4a88-a6b5-37fb8919f050 admin admin] PATH_INFO: `/v3/projects/bceff2dbe9f742508a2b57e512dc85c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:21.499942 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65e269-817d-4a88-a6b5-37fb8919f050 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bceff2dbe9f742508a2b57e512dc85c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:21.501932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65e269-817d-4a88-a6b5-37fb8919f050 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:21.503440 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bde0c-6973-4ad9-ac78-958c7b653ab4 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:21.508156 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a55f7-ca2b-4f6c-9497-20ac06132b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:21.508909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a55f7-ca2b-4f6c-9497-20ac06132b7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:21.509193 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a55f7-ca2b-4f6c-9497-20ac06132b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:21.509464 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a55f7-ca2b-4f6c-9497-20ac06132b7f admin admin] PATH_INFO: `/v3/projects/3bb6b5e60c9b4d08a2ce73c8c495c739` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:21.510603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a55f7-ca2b-4f6c-9497-20ac06132b7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bb6b5e60c9b4d08a2ce73c8c495c739)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:21.512772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a55f7-ca2b-4f6c-9497-20ac06132b7f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:21.637434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-5b2a55f7-ca2b-4f6c-9497-20ac06132b7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bb6b5e60c9b4d08a2ce73c8c495c739', 'request_id': 'req-5b2a55f7-ca2b-4f6c-9497-20ac06132b7f'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:21.639239 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-df65e269-817d-4a88-a6b5-37fb8919f050 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bceff2dbe9f742508a2b57e512dc85c1', 'request_id': 'req-df65e269-817d-4a88-a6b5-37fb8919f050'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:21.644381 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-5b2a55f7-ca2b-4f6c-9497-20ac06132b7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bb6b5e60c9b4d08a2ce73c8c495c739', 'request_id': 'req-5b2a55f7-ca2b-4f6c-9497-20ac06132b7f'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:21.715384 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-df65e269-817d-4a88-a6b5-37fb8919f050 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bceff2dbe9f742508a2b57e512dc85c1', 'request_id': 'req-df65e269-817d-4a88-a6b5-37fb8919f050'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:21.724948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-df65e269-817d-4a88-a6b5-37fb8919f050 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bceff2dbe9f742508a2b57e512dc85c1', 'request_id': 'req-df65e269-817d-4a88-a6b5-37fb8919f050'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:21.738439 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-df65e269-817d-4a88-a6b5-37fb8919f050 admin admin] The token cache is being invalidate because project bceff2dbe9f742508a2b57e512dc85c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:21.738740 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-df65e269-817d-4a88-a6b5-37fb8919f050 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:21.739703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-df65e269-817d-4a88-a6b5-37fb8919f050 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:21.741372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1028/2037] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:21 2022] DELETE /identity/v3/projects/bceff2dbe9f742508a2b57e512dc85c1 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:21.746563 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd993c82-a824-44db-87bd-b9c012557be3 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:21.746851 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd993c82-a824-44db-87bd-b9c012557be3 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:21.747133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd993c82-a824-44db-87bd-b9c012557be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:21.756724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:21.768959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-fd993c82-a824-44db-87bd-b9c012557be3 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:21.770784 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fd993c82-a824-44db-87bd-b9c012557be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:21.787628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-5b2a55f7-ca2b-4f6c-9497-20ac06132b7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bb6b5e60c9b4d08a2ce73c8c495c739', 'request_id': 'req-5b2a55f7-ca2b-4f6c-9497-20ac06132b7f'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:21.800335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-5b2a55f7-ca2b-4f6c-9497-20ac06132b7f admin admin] The token cache is being invalidate because project 3bb6b5e60c9b4d08a2ce73c8c495c739 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:21.800631 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-5b2a55f7-ca2b-4f6c-9497-20ac06132b7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:21.801284 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-5b2a55f7-ca2b-4f6c-9497-20ac06132b7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:21.803049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1010/2038] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:21 2022] DELETE /identity/v3/projects/3bb6b5e60c9b4d08a2ce73c8c495c739 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:21.807585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebd71b-83e3-4785-9f05-0c250b6b7909 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:21.811065 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebd71b-83e3-4785-9f05-0c250b6b7909 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:21.811065 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebd71b-83e3-4785-9f05-0c250b6b7909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:21.829766 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-ddebd71b-83e3-4785-9f05-0c250b6b7909 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:21.830990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ddebd71b-83e3-4785-9f05-0c250b6b7909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:21.833276 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1029/2039] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:21.841365 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb193ea-4aa0-42a5-a1f4-d533e080216b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:21.841677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb193ea-4aa0-42a5-a1f4-d533e080216b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:21.841966 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb193ea-4aa0-42a5-a1f4-d533e080216b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:21.844949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1011/2040] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:21.852488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894956a2-1671-4ed1-8f87-555e5b1b1cec None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:21.852772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894956a2-1671-4ed1-8f87-555e5b1b1cec None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:21.853041 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894956a2-1671-4ed1-8f87-555e5b1b1cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:21.865800 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-dcb193ea-4aa0-42a5-a1f4-d533e080216b None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:21.868733 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-dcb193ea-4aa0-42a5-a1f4-d533e080216b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:21.874509 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-894956a2-1671-4ed1-8f87-555e5b1b1cec None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:21.875743 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-894956a2-1671-4ed1-8f87-555e5b1b1cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:21.876313 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1030/2041] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:21.880592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb193ea-4aa0-42a5-a1f4-d533e080216b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:21.887591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1012/2042] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:21.891085 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb193ea-4aa0-42a5-a1f4-d533e080216b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:21.892269 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894956a2-1671-4ed1-8f87-555e5b1b1cec None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:21.898915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e2da9-311c-4f30-8b56-5c1886dc2aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:21.899821 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e2da9-311c-4f30-8b56-5c1886dc2aed admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:21.899986 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e2da9-311c-4f30-8b56-5c1886dc2aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:21.900143 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e2da9-311c-4f30-8b56-5c1886dc2aed admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:21.900809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e2da9-311c-4f30-8b56-5c1886dc2aed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:26:21.901117 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e2da9-311c-4f30-8b56-5c1886dc2aed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:21.902734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e2da9-311c-4f30-8b56-5c1886dc2aed admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:21.905133 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894956a2-1671-4ed1-8f87-555e5b1b1cec None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:21.912314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b80be2-c9a8-4798-9b6f-1a616184b0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:21.913602 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b80be2-c9a8-4798-9b6f-1a616184b0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:21.914193 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b80be2-c9a8-4798-9b6f-1a616184b0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:21.914499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b80be2-c9a8-4798-9b6f-1a616184b0b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:21.915558 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b80be2-c9a8-4798-9b6f-1a616184b0b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:26:21.916113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b80be2-c9a8-4798-9b6f-1a616184b0b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:21.919201 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b80be2-c9a8-4798-9b6f-1a616184b0b2 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:21.930247 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1031/2043] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:26:21.931794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e2da9-311c-4f30-8b56-5c1886dc2aed admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:21.941817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1013/2044] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:26:21.942702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b80be2-c9a8-4798-9b6f-1a616184b0b2 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:21.946686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e2da9-311c-4f30-8b56-5c1886dc2aed admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:21.952950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b80be2-c9a8-4798-9b6f-1a616184b0b2 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:21.955729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3632362-89f6-410a-83f4-e66f7b5e9a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:21.956355 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3632362-89f6-410a-83f4-e66f7b5e9a4b admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:21.956695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3632362-89f6-410a-83f4-e66f7b5e9a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:21.956914 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3632362-89f6-410a-83f4-e66f7b5e9a4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:21.957556 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3632362-89f6-410a-83f4-e66f7b5e9a4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:21.958366 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965084eb-ac9c-4069-a232-a684fa350944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:21.959054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965084eb-ac9c-4069-a232-a684fa350944 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:21.959445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965084eb-ac9c-4069-a232-a684fa350944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:21.959553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3632362-89f6-410a-83f4-e66f7b5e9a4b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:21.959794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965084eb-ac9c-4069-a232-a684fa350944 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:21.960517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965084eb-ac9c-4069-a232-a684fa350944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:21.962497 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965084eb-ac9c-4069-a232-a684fa350944 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:22.069885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1032/2045] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:21 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:26:22.073925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3632362-89f6-410a-83f4-e66f7b5e9a4b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:22.086971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1014/2046] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:21 2022] POST /identity/v3/projects => generated 367 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:26:22.091322 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965084eb-ac9c-4069-a232-a684fa350944 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:22.142992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3632362-89f6-410a-83f4-e66f7b5e9a4b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:22.146249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:22.157248 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965084eb-ac9c-4069-a232-a684fa350944 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:22.162438 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53531adb-14d7-4d98-83ed-18c018c725d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:22.163048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53531adb-14d7-4d98-83ed-18c018c725d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:22.163410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53531adb-14d7-4d98-83ed-18c018c725d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:22.163684 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53531adb-14d7-4d98-83ed-18c018c725d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:22.164313 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53531adb-14d7-4d98-83ed-18c018c725d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:22.166227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53531adb-14d7-4d98-83ed-18c018c725d4 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:22.201059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234fd9c-ae04-43ac-954d-4e21531a0405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:22.201673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234fd9c-ae04-43ac-954d-4e21531a0405 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:22.201953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234fd9c-ae04-43ac-954d-4e21531a0405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:22.202223 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234fd9c-ae04-43ac-954d-4e21531a0405 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:22.202843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234fd9c-ae04-43ac-954d-4e21531a0405 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:22.204780 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234fd9c-ae04-43ac-954d-4e21531a0405 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:22.361540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:22.369896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1015/2047] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:22 2022] POST /identity/v3/users => generated 284 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:26:22.370261 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1033/2048] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:22 2022] POST /identity/v3/users => generated 288 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:26:22.371618 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234fd9c-ae04-43ac-954d-4e21531a0405 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:22.372910 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53531adb-14d7-4d98-83ed-18c018c725d4 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:22.384841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234fd9c-ae04-43ac-954d-4e21531a0405 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:22.388803 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53531adb-14d7-4d98-83ed-18c018c725d4 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:22.391655 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f4622-2aea-4f23-80fc-edb7d91fe0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:22.396213 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f4622-2aea-4f23-80fc-edb7d91fe0dd admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:22.396213 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f4622-2aea-4f23-80fc-edb7d91fe0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:22.396213 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f4622-2aea-4f23-80fc-edb7d91fe0dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:22.396537 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476deb68-23d6-47b6-a0c4-762e6ef4a2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:22.396650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f4622-2aea-4f23-80fc-edb7d91fe0dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:22.396743 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476deb68-23d6-47b6-a0c4-762e6ef4a2fe admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:22.396743 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476deb68-23d6-47b6-a0c4-762e6ef4a2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:22.396743 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476deb68-23d6-47b6-a0c4-762e6ef4a2fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:22.396998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f4622-2aea-4f23-80fc-edb7d91fe0dd admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:22.397087 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476deb68-23d6-47b6-a0c4-762e6ef4a2fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:22.397245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476deb68-23d6-47b6-a0c4-762e6ef4a2fe admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:22.409811 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1034/2049] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:22.410030 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1016/2050] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:26:22.410879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f4622-2aea-4f23-80fc-edb7d91fe0dd admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:22.411749 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476deb68-23d6-47b6-a0c4-762e6ef4a2fe admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:22.421676 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f4622-2aea-4f23-80fc-edb7d91fe0dd admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:22.422339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476deb68-23d6-47b6-a0c4-762e6ef4a2fe admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:22.426568 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5012a60-b068-4f3c-8071-28aa8f3ce566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:22.427062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9633f53-855a-4fc5-aba2-6ea9e097dd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:22.427216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5012a60-b068-4f3c-8071-28aa8f3ce566 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:22.427496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5012a60-b068-4f3c-8071-28aa8f3ce566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:22.427712 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9633f53-855a-4fc5-aba2-6ea9e097dd64 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:22.427814 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5012a60-b068-4f3c-8071-28aa8f3ce566 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:22.427996 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9633f53-855a-4fc5-aba2-6ea9e097dd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:22.428273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9633f53-855a-4fc5-aba2-6ea9e097dd64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:22.428402 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5012a60-b068-4f3c-8071-28aa8f3ce566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:22.428916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9633f53-855a-4fc5-aba2-6ea9e097dd64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:22.429988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5012a60-b068-4f3c-8071-28aa8f3ce566 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:22.430495 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9633f53-855a-4fc5-aba2-6ea9e097dd64 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:22.445002 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1017/2051] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:22.445205 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1035/2052] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:22.446518 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5012a60-b068-4f3c-8071-28aa8f3ce566 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:22.447596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9633f53-855a-4fc5-aba2-6ea9e097dd64 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:22.456683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5012a60-b068-4f3c-8071-28aa8f3ce566 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:22.458032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9633f53-855a-4fc5-aba2-6ea9e097dd64 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:22.461449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be839fe4-1ddd-48aa-8ec6-f38dbfcc04c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:22.462127 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be839fe4-1ddd-48aa-8ec6-f38dbfcc04c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:22.462414 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be839fe4-1ddd-48aa-8ec6-f38dbfcc04c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:22.462694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be839fe4-1ddd-48aa-8ec6-f38dbfcc04c0 admin admin] PATH_INFO: `/v3/projects/57cb869af4804dadb729ea721433a386/users/51568d335e49402ebea68b9a315706e3/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:22.462972 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509876fc-1e74-4278-bd73-432af664dbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:22.463648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509876fc-1e74-4278-bd73-432af664dbcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:22.463929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509876fc-1e74-4278-bd73-432af664dbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:22.464496 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509876fc-1e74-4278-bd73-432af664dbcf admin admin] PATH_INFO: `/v3/projects/07d188f29f7948b2a7f3333b7a60bddc/users/e5abf021321f4c92b9fbe6e64965a6c1/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:22.478993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be839fe4-1ddd-48aa-8ec6-f38dbfcc04c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57cb869af4804dadb729ea721433a386, user_id=51568d335e49402ebea68b9a315706e3, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:22.479209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509876fc-1e74-4278-bd73-432af664dbcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=07d188f29f7948b2a7f3333b7a60bddc, user_id=e5abf021321f4c92b9fbe6e64965a6c1, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:22.482202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be839fe4-1ddd-48aa-8ec6-f38dbfcc04c0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:22.482650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509876fc-1e74-4278-bd73-432af664dbcf admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:22.577010 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1036/2053] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:22 2022] PUT /identity/v3/projects/57cb869af4804dadb729ea721433a386/users/51568d335e49402ebea68b9a315706e3/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:22.582452 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1018/2054] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:22 2022] PUT /identity/v3/projects/07d188f29f7948b2a7f3333b7a60bddc/users/e5abf021321f4c92b9fbe6e64965a6c1/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:22.587456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adf0e1-a9e9-4282-ab23-e00707c1f438 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:22.587742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adf0e1-a9e9-4282-ab23-e00707c1f438 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:22.593725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adf0e1-a9e9-4282-ab23-e00707c1f438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:22.593847 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e2b29-84e5-426c-9a18-8cb6ec6018a6 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:22.593847 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e2b29-84e5-426c-9a18-8cb6ec6018a6 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:22.593847 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e2b29-84e5-426c-9a18-8cb6ec6018a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:22.618330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-54adf0e1-a9e9-4282-ab23-e00707c1f438 None None] MFA Rules not processed for user `51568d335e49402ebea68b9a315706e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:22.619626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-54adf0e1-a9e9-4282-ab23-e00707c1f438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:22.621471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-626e2b29-84e5-426c-9a18-8cb6ec6018a6 None None] MFA Rules not processed for user `e5abf021321f4c92b9fbe6e64965a6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:22.622683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-626e2b29-84e5-426c-9a18-8cb6ec6018a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:22.686348 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1037/2055] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:22.688367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1019/2056] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:22.690886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54adf0e1-a9e9-4282-ab23-e00707c1f438 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:22.699155 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e2b29-84e5-426c-9a18-8cb6ec6018a6 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:22.700521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-626e2b29-84e5-426c-9a18-8cb6ec6018a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:22.753390 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54adf0e1-a9e9-4282-ab23-e00707c1f438 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:22.757954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e2b29-84e5-426c-9a18-8cb6ec6018a6 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:22.758116 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a30b74-f296-491b-805e-1c5d58a36f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:22.758628 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a30b74-f296-491b-805e-1c5d58a36f5b admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:22.758909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a30b74-f296-491b-805e-1c5d58a36f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:22.759193 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a30b74-f296-491b-805e-1c5d58a36f5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:22.759832 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a30b74-f296-491b-805e-1c5d58a36f5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:22.762251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb724764-1545-4971-849a-67392d5fb452 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:22.762389 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a30b74-f296-491b-805e-1c5d58a36f5b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:22.762866 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb724764-1545-4971-849a-67392d5fb452 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:22.763151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb724764-1545-4971-849a-67392d5fb452 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:22.763531 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb724764-1545-4971-849a-67392d5fb452 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:22.773083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb724764-1545-4971-849a-67392d5fb452 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:22.775297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb724764-1545-4971-849a-67392d5fb452 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:22.786262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1020/2057] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:26:22.910254 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1038/2058] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:22 2022] POST /identity/v3/projects => generated 369 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:26:22.911134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb724764-1545-4971-849a-67392d5fb452 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:22.924838 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb724764-1545-4971-849a-67392d5fb452 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:22.958165 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:22.993380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486eb44-fcec-4339-b187-2372d40fccfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:22.995954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486eb44-fcec-4339-b187-2372d40fccfb admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:22.995954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486eb44-fcec-4339-b187-2372d40fccfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:22.995954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486eb44-fcec-4339-b187-2372d40fccfb admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:22.995954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486eb44-fcec-4339-b187-2372d40fccfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:22.997716 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486eb44-fcec-4339-b187-2372d40fccfb admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:23.077729 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1021/2059] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:22 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:23.086330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a30b74-f296-491b-805e-1c5d58a36f5b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:23.098726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a30b74-f296-491b-805e-1c5d58a36f5b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:23.113573 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a57027-acae-4463-a000-4b82db9dc804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:23.114483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a57027-acae-4463-a000-4b82db9dc804 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:23.114903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a57027-acae-4463-a000-4b82db9dc804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:23.115029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a57027-acae-4463-a000-4b82db9dc804 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:23.115722 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a57027-acae-4463-a000-4b82db9dc804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:23.117337 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a57027-acae-4463-a000-4b82db9dc804 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:23.136052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1039/2060] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:23.140916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486eb44-fcec-4339-b187-2372d40fccfb admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:23.151307 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486eb44-fcec-4339-b187-2372d40fccfb admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:23.160649 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2aaf8-8bba-4326-ad5e-d0fd935b95d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:23.161497 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2aaf8-8bba-4326-ad5e-d0fd935b95d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:23.161790 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2aaf8-8bba-4326-ad5e-d0fd935b95d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:23.162066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2aaf8-8bba-4326-ad5e-d0fd935b95d2 admin admin] PATH_INFO: `/v3/projects/4b50d22a985b485e9c5508ccef7d4cc1/users/8dd590d2629e448ab168b09905f278a4/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:23.179497 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2aaf8-8bba-4326-ad5e-d0fd935b95d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b50d22a985b485e9c5508ccef7d4cc1, user_id=8dd590d2629e448ab168b09905f278a4, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:23.182175 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2aaf8-8bba-4326-ad5e-d0fd935b95d2 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:23.297703 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1022/2061] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:23 2022] PUT /identity/v3/projects/4b50d22a985b485e9c5508ccef7d4cc1/users/8dd590d2629e448ab168b09905f278a4/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:23.301912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a57027-acae-4463-a000-4b82db9dc804 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:23.314766 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a57027-acae-4463-a000-4b82db9dc804 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:23.329433 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:23.391211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4edbb-477a-483b-a4a8-1f04dfbfbe4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:23.392141 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4edbb-477a-483b-a4a8-1f04dfbfbe4f admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:23.392795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4edbb-477a-483b-a4a8-1f04dfbfbe4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:23.393140 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4edbb-477a-483b-a4a8-1f04dfbfbe4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:23.393801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4edbb-477a-483b-a4a8-1f04dfbfbe4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:23.395462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4edbb-477a-483b-a4a8-1f04dfbfbe4f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:23.409995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1040/2062] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:23.413615 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2aaf8-8bba-4326-ad5e-d0fd935b95d2 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:23.425014 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2aaf8-8bba-4326-ad5e-d0fd935b95d2 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:23.434157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c774234-85a9-442c-aaad-14d3ae8540e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:23.434157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c774234-85a9-442c-aaad-14d3ae8540e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:23.434512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c774234-85a9-442c-aaad-14d3ae8540e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:23.434512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c774234-85a9-442c-aaad-14d3ae8540e3 admin admin] PATH_INFO: `/v3/projects/4b50d22a985b485e9c5508ccef7d4cc1/users/8dd590d2629e448ab168b09905f278a4/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:23.436531 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c774234-85a9-442c-aaad-14d3ae8540e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b50d22a985b485e9c5508ccef7d4cc1, user_id=8dd590d2629e448ab168b09905f278a4, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:23.439413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c774234-85a9-442c-aaad-14d3ae8540e3 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:23.450715 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1023/2063] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:23 2022] PUT /identity/v3/projects/4b50d22a985b485e9c5508ccef7d4cc1/users/8dd590d2629e448ab168b09905f278a4/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:23.468295 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2939e-15ec-4c3e-934b-7a9c9f5937fd None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:23.468586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2939e-15ec-4c3e-934b-7a9c9f5937fd None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:23.468866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2939e-15ec-4c3e-934b-7a9c9f5937fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:23.500282 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-25e2939e-15ec-4c3e-934b-7a9c9f5937fd None None] MFA Rules not processed for user `8dd590d2629e448ab168b09905f278a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:23.501543 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-25e2939e-15ec-4c3e-934b-7a9c9f5937fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:23.580067 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1041/2064] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:23.598630 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c774234-85a9-442c-aaad-14d3ae8540e3 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:23.600500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5c774234-85a9-442c-aaad-14d3ae8540e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:23.669061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c774234-85a9-442c-aaad-14d3ae8540e3 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:23.673315 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35597c6d-740c-4bb5-b3be-ebeef29e06b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:23.674171 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35597c6d-740c-4bb5-b3be-ebeef29e06b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:23.674494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35597c6d-740c-4bb5-b3be-ebeef29e06b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:23.674863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35597c6d-740c-4bb5-b3be-ebeef29e06b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:23.688018 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35597c6d-740c-4bb5-b3be-ebeef29e06b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:23.691913 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35597c6d-740c-4bb5-b3be-ebeef29e06b8 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:23.705485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1024/2065] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:26:23.828143 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2939e-15ec-4c3e-934b-7a9c9f5937fd None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:23.837826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2939e-15ec-4c3e-934b-7a9c9f5937fd None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:23.842470 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdbb74-65b2-46a9-8458-1079c359127e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:23.843095 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdbb74-65b2-46a9-8458-1079c359127e service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:23.846662 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdbb74-65b2-46a9-8458-1079c359127e service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:23.846662 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdbb74-65b2-46a9-8458-1079c359127e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:23.908954 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdbb74-65b2-46a9-8458-1079c359127e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:23.910388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdbb74-65b2-46a9-8458-1079c359127e service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:23.919663 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1042/2066] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:29.977469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35597c6d-740c-4bb5-b3be-ebeef29e06b8 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:30.041547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35597c6d-740c-4bb5-b3be-ebeef29e06b8 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:30.046506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6aa51-6585-41b8-aa25-4151b6297ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:30.047127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6aa51-6585-41b8-aa25-4151b6297ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:30.047442 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6aa51-6585-41b8-aa25-4151b6297ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:30.047709 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6aa51-6585-41b8-aa25-4151b6297ea4 admin admin] PATH_INFO: `/v3/users/51568d335e49402ebea68b9a315706e3` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:30.048927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6aa51-6585-41b8-aa25-4151b6297ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51568d335e49402ebea68b9a315706e3)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:30.050998 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6aa51-6585-41b8-aa25-4151b6297ea4 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:30.165897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:30.318344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-29b6aa51-6585-41b8-aa25-4151b6297ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51568d335e49402ebea68b9a315706e3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:30.332556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-29b6aa51-6585-41b8-aa25-4151b6297ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51568d335e49402ebea68b9a315706e3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:30.342162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-29b6aa51-6585-41b8-aa25-4151b6297ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51568d335e49402ebea68b9a315706e3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:30.496550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-29b6aa51-6585-41b8-aa25-4151b6297ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51568d335e49402ebea68b9a315706e3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:30.497194 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-29b6aa51-6585-41b8-aa25-4151b6297ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51568d335e49402ebea68b9a315706e3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:30.503534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1025/2067] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:29 2022] DELETE /identity/v3/users/51568d335e49402ebea68b9a315706e3 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:30.515011 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759543-4bcc-40e2-bd91-1174787d837d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:30.515571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759543-4bcc-40e2-bd91-1174787d837d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:30.515712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759543-4bcc-40e2-bd91-1174787d837d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:30.535147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f7759543-4bcc-40e2-bd91-1174787d837d None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:30.536449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f7759543-4bcc-40e2-bd91-1174787d837d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:30.577297 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:30.619769 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1043/2068] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:30.631070 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6aa51-6585-41b8-aa25-4151b6297ea4 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:30.632734 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-29b6aa51-6585-41b8-aa25-4151b6297ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:30.692458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6aa51-6585-41b8-aa25-4151b6297ea4 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:30.696414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837f61e-4e7a-4c1a-b395-a6929a2d45c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:30.697053 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837f61e-4e7a-4c1a-b395-a6929a2d45c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:30.697332 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837f61e-4e7a-4c1a-b395-a6929a2d45c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:30.697604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837f61e-4e7a-4c1a-b395-a6929a2d45c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:30.710491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837f61e-4e7a-4c1a-b395-a6929a2d45c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:30.710491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837f61e-4e7a-4c1a-b395-a6929a2d45c9 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:30.718992 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1026/2069] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:30.780505 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7759543-4bcc-40e2-bd91-1174787d837d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:30.782099 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f7759543-4bcc-40e2-bd91-1174787d837d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:30.795967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7759543-4bcc-40e2-bd91-1174787d837d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:30.804519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40811515-5261-4b5f-a685-fbfc363015ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:30.804519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40811515-5261-4b5f-a685-fbfc363015ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:30.804519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40811515-5261-4b5f-a685-fbfc363015ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:30.804519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40811515-5261-4b5f-a685-fbfc363015ae admin admin] PATH_INFO: `/v3/users/22a236ae660741019d8e47f4e4e64e41` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:30.804855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40811515-5261-4b5f-a685-fbfc363015ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=22a236ae660741019d8e47f4e4e64e41)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:30.807691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40811515-5261-4b5f-a685-fbfc363015ae admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:31.078170 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-40811515-5261-4b5f-a685-fbfc363015ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22a236ae660741019d8e47f4e4e64e41'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:31.096028 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-40811515-5261-4b5f-a685-fbfc363015ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22a236ae660741019d8e47f4e4e64e41'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:31.108273 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-40811515-5261-4b5f-a685-fbfc363015ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22a236ae660741019d8e47f4e4e64e41'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:31.227923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-40811515-5261-4b5f-a685-fbfc363015ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22a236ae660741019d8e47f4e4e64e41'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:31.228666 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-40811515-5261-4b5f-a685-fbfc363015ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22a236ae660741019d8e47f4e4e64e41'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:31.236757 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1044/2070] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:30 2022] DELETE /identity/v3/users/22a236ae660741019d8e47f4e4e64e41 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:31.241420 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8daea5-33cf-4dc7-917b-ce2893494ce1 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:31.241905 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8daea5-33cf-4dc7-917b-ce2893494ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:31.242007 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8daea5-33cf-4dc7-917b-ce2893494ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:31.279012 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40811515-5261-4b5f-a685-fbfc363015ae admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:31.280773 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-8e8daea5-33cf-4dc7-917b-ce2893494ce1 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:31.282105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-40811515-5261-4b5f-a685-fbfc363015ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:31.287050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-8e8daea5-33cf-4dc7-917b-ce2893494ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:31.437682 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40811515-5261-4b5f-a685-fbfc363015ae admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:31.442820 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:31.443578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:31.443866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:31.444207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76 admin admin] PATH_INFO: `/v3/projects/57cb869af4804dadb729ea721433a386` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:31.445359 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57cb869af4804dadb729ea721433a386)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:31.447461 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:31.475870 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1027/2071] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:31.488657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8daea5-33cf-4dc7-917b-ce2893494ce1 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:31.492571 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-8e8daea5-33cf-4dc7-917b-ce2893494ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:31.594009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:31.606407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8daea5-33cf-4dc7-917b-ce2893494ce1 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:31.613198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc263d-79f6-45ee-b8f5-c8885fdcbd80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:31.614134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc263d-79f6-45ee-b8f5-c8885fdcbd80 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:31.614602 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc263d-79f6-45ee-b8f5-c8885fdcbd80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:31.615067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc263d-79f6-45ee-b8f5-c8885fdcbd80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:31.660357 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57cb869af4804dadb729ea721433a386', 'request_id': 'req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:31.665446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc263d-79f6-45ee-b8f5-c8885fdcbd80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:31.667402 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:31.668370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc263d-79f6-45ee-b8f5-c8885fdcbd80 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:31.692950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1028/2072] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:31.768090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57cb869af4804dadb729ea721433a386', 'request_id': 'req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:31.795382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57cb869af4804dadb729ea721433a386', 'request_id': 'req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:31.812471 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76 admin admin] The token cache is being invalidate because project 57cb869af4804dadb729ea721433a386 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:31.812768 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:31.813452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:31.815155 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1045/2073] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:31 2022] DELETE /identity/v3/projects/57cb869af4804dadb729ea721433a386 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:31.819300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc263d-79f6-45ee-b8f5-c8885fdcbd80 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:31.830462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc263d-79f6-45ee-b8f5-c8885fdcbd80 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:31.903402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b650505-0176-40d2-a951-7521f01253c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:31.904116 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b650505-0176-40d2-a951-7521f01253c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:31.904465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b650505-0176-40d2-a951-7521f01253c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:31.904809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b650505-0176-40d2-a951-7521f01253c2 admin admin] PATH_INFO: `/v3/users/8dd590d2629e448ab168b09905f278a4` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:31.906013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b650505-0176-40d2-a951-7521f01253c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dd590d2629e448ab168b09905f278a4)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:31.908028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b650505-0176-40d2-a951-7521f01253c2 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:32.164086 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-4b650505-0176-40d2-a951-7521f01253c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dd590d2629e448ab168b09905f278a4'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:32.179040 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-4b650505-0176-40d2-a951-7521f01253c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dd590d2629e448ab168b09905f278a4'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:32.188673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-4b650505-0176-40d2-a951-7521f01253c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dd590d2629e448ab168b09905f278a4'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:32.320646 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-4b650505-0176-40d2-a951-7521f01253c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dd590d2629e448ab168b09905f278a4'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:32.321434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-4b650505-0176-40d2-a951-7521f01253c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dd590d2629e448ab168b09905f278a4'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:32.327269 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:32.333576 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:32.333997 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1029/2074] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:31 2022] DELETE /identity/v3/users/8dd590d2629e448ab168b09905f278a4 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:32.357305 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9023efb-8e0c-44b9-a1ed-6dd6c299ab76 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:32.476263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef43dd-0579-4a9e-98cd-3591ae29edd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:32.477078 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef43dd-0579-4a9e-98cd-3591ae29edd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:32.477606 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef43dd-0579-4a9e-98cd-3591ae29edd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:32.478071 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef43dd-0579-4a9e-98cd-3591ae29edd2 admin admin] PATH_INFO: `/v3/projects/2fe22248259d41beb1e7018ea86e5d9a` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:32.479894 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef43dd-0579-4a9e-98cd-3591ae29edd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fe22248259d41beb1e7018ea86e5d9a)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:32.482473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef43dd-0579-4a9e-98cd-3591ae29edd2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:32.627893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4aef43dd-0579-4a9e-98cd-3591ae29edd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fe22248259d41beb1e7018ea86e5d9a', 'request_id': 'req-4aef43dd-0579-4a9e-98cd-3591ae29edd2'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:32.773402 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4aef43dd-0579-4a9e-98cd-3591ae29edd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fe22248259d41beb1e7018ea86e5d9a', 'request_id': 'req-4aef43dd-0579-4a9e-98cd-3591ae29edd2'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:32.790250 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4aef43dd-0579-4a9e-98cd-3591ae29edd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fe22248259d41beb1e7018ea86e5d9a', 'request_id': 'req-4aef43dd-0579-4a9e-98cd-3591ae29edd2'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:32.807110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4aef43dd-0579-4a9e-98cd-3591ae29edd2 admin admin] The token cache is being invalidate because project 2fe22248259d41beb1e7018ea86e5d9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:32.807648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4aef43dd-0579-4a9e-98cd-3591ae29edd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:32.808613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4aef43dd-0579-4a9e-98cd-3591ae29edd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:32.810944 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1046/2075] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:32 2022] DELETE /identity/v3/projects/2fe22248259d41beb1e7018ea86e5d9a => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:26:32.814728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b650505-0176-40d2-a951-7521f01253c2 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:32.817848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4b650505-0176-40d2-a951-7521f01253c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:32.834958 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b650505-0176-40d2-a951-7521f01253c2 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:32.973758 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24e011-487f-465a-8326-25c4bd4fb1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:32.974720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24e011-487f-465a-8326-25c4bd4fb1d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:32.976283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24e011-487f-465a-8326-25c4bd4fb1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:32.976283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24e011-487f-465a-8326-25c4bd4fb1d2 admin admin] PATH_INFO: `/v3/users/5f8e7973285b4583bb4fe3039ba49636` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:32.977102 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24e011-487f-465a-8326-25c4bd4fb1d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f8e7973285b4583bb4fe3039ba49636)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:32.978938 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24e011-487f-465a-8326-25c4bd4fb1d2 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:32.990603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:33.222637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ca24e011-487f-465a-8326-25c4bd4fb1d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f8e7973285b4583bb4fe3039ba49636'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:33.250145 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ca24e011-487f-465a-8326-25c4bd4fb1d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f8e7973285b4583bb4fe3039ba49636'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:33.274369 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef43dd-0579-4a9e-98cd-3591ae29edd2 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:33.276500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4aef43dd-0579-4a9e-98cd-3591ae29edd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:33.278550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ca24e011-487f-465a-8326-25c4bd4fb1d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f8e7973285b4583bb4fe3039ba49636'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:33.309716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef43dd-0579-4a9e-98cd-3591ae29edd2 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:33.313232 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6746b-43e0-4b17-92a6-f15c92db4ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:33.314065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6746b-43e0-4b17-92a6-f15c92db4ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:33.314517 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6746b-43e0-4b17-92a6-f15c92db4ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:33.314927 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6746b-43e0-4b17-92a6-f15c92db4ad4 admin admin] PATH_INFO: `/v3/projects/4b50d22a985b485e9c5508ccef7d4cc1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:33.317138 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6746b-43e0-4b17-92a6-f15c92db4ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b50d22a985b485e9c5508ccef7d4cc1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:33.319988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6746b-43e0-4b17-92a6-f15c92db4ad4 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:33.404807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ca24e011-487f-465a-8326-25c4bd4fb1d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f8e7973285b4583bb4fe3039ba49636'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:33.405669 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ca24e011-487f-465a-8326-25c4bd4fb1d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f8e7973285b4583bb4fe3039ba49636'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:33.413942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1030/2076] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:32 2022] DELETE /identity/v3/users/5f8e7973285b4583bb4fe3039ba49636 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:26:33.543469 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-83d6746b-43e0-4b17-92a6-f15c92db4ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b50d22a985b485e9c5508ccef7d4cc1', 'request_id': 'req-83d6746b-43e0-4b17-92a6-f15c92db4ad4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:33.675110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-83d6746b-43e0-4b17-92a6-f15c92db4ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b50d22a985b485e9c5508ccef7d4cc1', 'request_id': 'req-83d6746b-43e0-4b17-92a6-f15c92db4ad4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:33.684578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-83d6746b-43e0-4b17-92a6-f15c92db4ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b50d22a985b485e9c5508ccef7d4cc1', 'request_id': 'req-83d6746b-43e0-4b17-92a6-f15c92db4ad4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:33.691242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:33.700124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-83d6746b-43e0-4b17-92a6-f15c92db4ad4 admin admin] The token cache is being invalidate because project 4b50d22a985b485e9c5508ccef7d4cc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:33.700715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-83d6746b-43e0-4b17-92a6-f15c92db4ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:33.701595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-83d6746b-43e0-4b17-92a6-f15c92db4ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:33.703147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1047/2077] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:33 2022] DELETE /identity/v3/projects/4b50d22a985b485e9c5508ccef7d4cc1 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:33.709432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e7ad2-f9b9-49ba-8896-fb0da3d3ae53 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:33.709701 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e7ad2-f9b9-49ba-8896-fb0da3d3ae53 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:33.709973 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e7ad2-f9b9-49ba-8896-fb0da3d3ae53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:33.732405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-2a1e7ad2-f9b9-49ba-8896-fb0da3d3ae53 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:33.734035 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2a1e7ad2-f9b9-49ba-8896-fb0da3d3ae53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:33.849878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1031/2078] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:33.850996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e64f81-5771-4f36-947a-cb9d1d6bf382 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:33.851485 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e64f81-5771-4f36-947a-cb9d1d6bf382 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:33.851585 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e64f81-5771-4f36-947a-cb9d1d6bf382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:33.892580 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-35e64f81-5771-4f36-947a-cb9d1d6bf382 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:33.894894 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-35e64f81-5771-4f36-947a-cb9d1d6bf382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:33.921891 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1048/2079] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:33.926267 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e7ad2-f9b9-49ba-8896-fb0da3d3ae53 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:33.937836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e7ad2-f9b9-49ba-8896-fb0da3d3ae53 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:33.945170 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd8d12-cd2f-42d1-b963-db14c7f8f656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:33.946088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd8d12-cd2f-42d1-b963-db14c7f8f656 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:33.946459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd8d12-cd2f-42d1-b963-db14c7f8f656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:33.946820 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd8d12-cd2f-42d1-b963-db14c7f8f656 admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:33.948357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd8d12-cd2f-42d1-b963-db14c7f8f656 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:26:33.948720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd8d12-cd2f-42d1-b963-db14c7f8f656 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:33.950854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd8d12-cd2f-42d1-b963-db14c7f8f656 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:33.966762 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1032/2080] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:26:33.971257 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e64f81-5771-4f36-947a-cb9d1d6bf382 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:33.994043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e64f81-5771-4f36-947a-cb9d1d6bf382 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:34.009033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e68e6c-c0fd-42a7-abf7-bb3be3116527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:34.009734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e68e6c-c0fd-42a7-abf7-bb3be3116527 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:34.010088 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e68e6c-c0fd-42a7-abf7-bb3be3116527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:34.010432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e68e6c-c0fd-42a7-abf7-bb3be3116527 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:34.011194 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e68e6c-c0fd-42a7-abf7-bb3be3116527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:34.013165 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e68e6c-c0fd-42a7-abf7-bb3be3116527 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:34.243324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd8d12-cd2f-42d1-b963-db14c7f8f656 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:34.251224 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1049/2081] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:33 2022] POST /identity/v3/projects => generated 407 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:34.257398 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd8d12-cd2f-42d1-b963-db14c7f8f656 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:34.419032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b30701-4b00-4fd6-b831-c345ca918524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:34.421051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b30701-4b00-4fd6-b831-c345ca918524 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:34.421526 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b30701-4b00-4fd6-b831-c345ca918524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:34.421955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b30701-4b00-4fd6-b831-c345ca918524 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:34.422756 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b30701-4b00-4fd6-b831-c345ca918524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:34.426100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b30701-4b00-4fd6-b831-c345ca918524 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:34.531906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e68e6c-c0fd-42a7-abf7-bb3be3116527 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:34.531906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-70e68e6c-c0fd-42a7-abf7-bb3be3116527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:34.544886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e68e6c-c0fd-42a7-abf7-bb3be3116527 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:34.554837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f510c1-01a7-4f79-b6aa-15f403ceffb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:34.555802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f510c1-01a7-4f79-b6aa-15f403ceffb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:34.556424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f510c1-01a7-4f79-b6aa-15f403ceffb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:34.557515 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f510c1-01a7-4f79-b6aa-15f403ceffb3 admin admin] PATH_INFO: `/v3/projects/4b157b74e6144c3f9c10a8016bf684f9` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:34.559530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f510c1-01a7-4f79-b6aa-15f403ceffb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b157b74e6144c3f9c10a8016bf684f9)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:34.575940 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1033/2082] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:34 2022] POST /identity/v3/users => generated 304 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:26:34.576879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f510c1-01a7-4f79-b6aa-15f403ceffb3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:34.577848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b30701-4b00-4fd6-b831-c345ca918524 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:34.592401 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b30701-4b00-4fd6-b831-c345ca918524 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:34.600539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcbba2-a0c4-4160-8cb4-510b3a96d460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:34.600539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcbba2-a0c4-4160-8cb4-510b3a96d460 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:34.600539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcbba2-a0c4-4160-8cb4-510b3a96d460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:34.600539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcbba2-a0c4-4160-8cb4-510b3a96d460 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:34.601618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddcbba2-a0c4-4160-8cb4-510b3a96d460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:34.603371 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddcbba2-a0c4-4160-8cb4-510b3a96d460 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:34.616491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1034/2083] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:26:34.620887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcbba2-a0c4-4160-8cb4-510b3a96d460 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:34.632886 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcbba2-a0c4-4160-8cb4-510b3a96d460 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:34.640640 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba0d24-db32-4f5c-9742-9073b44c4844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:34.641837 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba0d24-db32-4f5c-9742-9073b44c4844 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:34.642182 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba0d24-db32-4f5c-9742-9073b44c4844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:34.642536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba0d24-db32-4f5c-9742-9073b44c4844 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:34.643277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba0d24-db32-4f5c-9742-9073b44c4844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:34.645043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba0d24-db32-4f5c-9742-9073b44c4844 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:34.659475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1035/2084] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:26:34.661008 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba0d24-db32-4f5c-9742-9073b44c4844 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:34.670981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba0d24-db32-4f5c-9742-9073b44c4844 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:34.675978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b8d5c-93ae-463f-b524-b38bd63b8442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:34.676635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b8d5c-93ae-463f-b524-b38bd63b8442 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:34.676939 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b8d5c-93ae-463f-b524-b38bd63b8442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:34.677211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b8d5c-93ae-463f-b524-b38bd63b8442 admin admin] PATH_INFO: `/v3/projects/12154033004241e4b957c313b74d8667/users/ad171360ac174f0f9fe3f7f6579d0836/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:34.692889 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b8d5c-93ae-463f-b524-b38bd63b8442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12154033004241e4b957c313b74d8667, user_id=ad171360ac174f0f9fe3f7f6579d0836, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:34.695534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b8d5c-93ae-463f-b524-b38bd63b8442 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:34.787115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1036/2085] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:34 2022] PUT /identity/v3/projects/12154033004241e4b957c313b74d8667/users/ad171360ac174f0f9fe3f7f6579d0836/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:34.793569 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-91f510c1-01a7-4f79-b6aa-15f403ceffb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b157b74e6144c3f9c10a8016bf684f9', 'request_id': 'req-91f510c1-01a7-4f79-b6aa-15f403ceffb3'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:34.799311 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b482e-e51b-4518-a878-57bc559e33cf None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:34.799685 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b482e-e51b-4518-a878-57bc559e33cf None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:34.800070 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b482e-e51b-4518-a878-57bc559e33cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:34.831620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-898b482e-e51b-4518-a878-57bc559e33cf None None] MFA Rules not processed for user `ad171360ac174f0f9fe3f7f6579d0836`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:34.832878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-898b482e-e51b-4518-a878-57bc559e33cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:34.929217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1037/2086] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:34 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:34.930781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b482e-e51b-4518-a878-57bc559e33cf None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:34.954833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-91f510c1-01a7-4f79-b6aa-15f403ceffb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b157b74e6144c3f9c10a8016bf684f9', 'request_id': 'req-91f510c1-01a7-4f79-b6aa-15f403ceffb3'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:34.965965 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-91f510c1-01a7-4f79-b6aa-15f403ceffb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b157b74e6144c3f9c10a8016bf684f9', 'request_id': 'req-91f510c1-01a7-4f79-b6aa-15f403ceffb3'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:34.994185 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-91f510c1-01a7-4f79-b6aa-15f403ceffb3 admin admin] The token cache is being invalidate because project 4b157b74e6144c3f9c10a8016bf684f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:34.994751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-91f510c1-01a7-4f79-b6aa-15f403ceffb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:34.996132 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-91f510c1-01a7-4f79-b6aa-15f403ceffb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:35.015922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1050/2087] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:34 2022] DELETE /identity/v3/projects/4b157b74e6144c3f9c10a8016bf684f9 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:26:35.016160 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b482e-e51b-4518-a878-57bc559e33cf None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:35.017668 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaec9f3-388e-4fea-9355-3fdeccb79102 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.018021 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaec9f3-388e-4fea-9355-3fdeccb79102 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.018357 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaec9f3-388e-4fea-9355-3fdeccb79102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.022020 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546619d-661e-4129-be60-621e5f009fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:35.022647 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546619d-661e-4129-be60-621e5f009fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.022923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546619d-661e-4129-be60-621e5f009fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.023220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546619d-661e-4129-be60-621e5f009fc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.023858 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546619d-661e-4129-be60-621e5f009fc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:35.025749 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546619d-661e-4129-be60-621e5f009fc9 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:35.045600 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-faaec9f3-388e-4fea-9355-3fdeccb79102 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:35.046923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-faaec9f3-388e-4fea-9355-3fdeccb79102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:35.055386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1051/2088] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:35.063570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f7961-24d4-44e2-8e0d-1daeae08d1ee None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.063978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f7961-24d4-44e2-8e0d-1daeae08d1ee None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.064321 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f7961-24d4-44e2-8e0d-1daeae08d1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.097962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ff6f7961-24d4-44e2-8e0d-1daeae08d1ee None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:35.097962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ff6f7961-24d4-44e2-8e0d-1daeae08d1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:35.109084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1052/2089] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:35.114372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:35.116911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f7961-24d4-44e2-8e0d-1daeae08d1ee None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:35.135994 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f7961-24d4-44e2-8e0d-1daeae08d1ee None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:35.138953 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1038/2090] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:34 2022] POST /identity/v3/projects => generated 409 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:35.141664 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81223f-c774-4d31-8d86-4a4912396e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:35.142497 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81223f-c774-4d31-8d86-4a4912396e9a admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.142788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81223f-c774-4d31-8d86-4a4912396e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.143068 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81223f-c774-4d31-8d86-4a4912396e9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.143656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546619d-661e-4129-be60-621e5f009fc9 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:35.144037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c81223f-c774-4d31-8d86-4a4912396e9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:26:35.144347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c81223f-c774-4d31-8d86-4a4912396e9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:35.146256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-6546619d-661e-4129-be60-621e5f009fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:35.146368 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c81223f-c774-4d31-8d86-4a4912396e9a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:35.184834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1053/2091] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:26:35.186150 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81223f-c774-4d31-8d86-4a4912396e9a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:35.204662 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81223f-c774-4d31-8d86-4a4912396e9a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:35.246477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546619d-661e-4129-be60-621e5f009fc9 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:35.251271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb781b01-5479-4c75-98e1-a66633b954a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:35.251887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb781b01-5479-4c75-98e1-a66633b954a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.252168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb781b01-5479-4c75-98e1-a66633b954a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.252448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb781b01-5479-4c75-98e1-a66633b954a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.253092 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb781b01-5479-4c75-98e1-a66633b954a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:35.255137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb781b01-5479-4c75-98e1-a66633b954a8 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:35.270498 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5ca61-0e2b-45a0-a310-00e51ac3f6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:35.272958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5ca61-0e2b-45a0-a310-00e51ac3f6d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.272958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5ca61-0e2b-45a0-a310-00e51ac3f6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.272958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5ca61-0e2b-45a0-a310-00e51ac3f6d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.272958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5ca61-0e2b-45a0-a310-00e51ac3f6d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:35.274292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5ca61-0e2b-45a0-a310-00e51ac3f6d7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:35.405232 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1039/2092] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:35 2022] POST /identity/v3/users => generated 311 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:35.409304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb781b01-5479-4c75-98e1-a66633b954a8 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:35.420846 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb781b01-5479-4c75-98e1-a66633b954a8 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:35.427483 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9d2e0-3b07-4e25-9e4d-c85cf300ecf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:35.428183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9d2e0-3b07-4e25-9e4d-c85cf300ecf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.428467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9d2e0-3b07-4e25-9e4d-c85cf300ecf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.428741 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9d2e0-3b07-4e25-9e4d-c85cf300ecf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.429380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9d2e0-3b07-4e25-9e4d-c85cf300ecf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:35.431538 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9d2e0-3b07-4e25-9e4d-c85cf300ecf5 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:35.454454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1054/2093] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:35 2022] POST /identity/v3/projects => generated 391 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:35.456357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1040/2094] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:26:35.457255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5ca61-0e2b-45a0-a310-00e51ac3f6d7 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:35.459083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9d2e0-3b07-4e25-9e4d-c85cf300ecf5 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:35.475386 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9d2e0-3b07-4e25-9e4d-c85cf300ecf5 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:35.535260 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5ca61-0e2b-45a0-a310-00e51ac3f6d7 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:35.540211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb118a-4619-4c19-bd0b-80b85f63066a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:35.540885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb118a-4619-4c19-bd0b-80b85f63066a admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.541262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb118a-4619-4c19-bd0b-80b85f63066a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.541523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb118a-4619-4c19-bd0b-80b85f63066a admin admin] PATH_INFO: `/v3/projects/069464e6363946ef9efc3e1145c524a6/users/59fe590a4b744e1c8270d968773ee77a/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.552385 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148cedce-7b1b-4c70-8d75-efc6ec08504e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:35.553091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cedce-7b1b-4c70-8d75-efc6ec08504e admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.553607 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cedce-7b1b-4c70-8d75-efc6ec08504e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.553883 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cedce-7b1b-4c70-8d75-efc6ec08504e admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.554555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148cedce-7b1b-4c70-8d75-efc6ec08504e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:35.556579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148cedce-7b1b-4c70-8d75-efc6ec08504e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:35.564137 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb118a-4619-4c19-bd0b-80b85f63066a admin admin] RBAC: Authorizing `identity:create_grant(project_id=069464e6363946ef9efc3e1145c524a6, user_id=59fe590a4b744e1c8270d968773ee77a, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:35.566946 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb118a-4619-4c19-bd0b-80b85f63066a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:35.675683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1055/2095] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:35 2022] PUT /identity/v3/projects/069464e6363946ef9efc3e1145c524a6/users/59fe590a4b744e1c8270d968773ee77a/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:26:35.679667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb118a-4619-4c19-bd0b-80b85f63066a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:35.697997 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1041/2096] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:35 2022] POST /identity/v3/users => generated 296 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:26:35.702824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148cedce-7b1b-4c70-8d75-efc6ec08504e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:35.715313 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148cedce-7b1b-4c70-8d75-efc6ec08504e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:35.755912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb118a-4619-4c19-bd0b-80b85f63066a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:35.760872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86785fc7-123a-4c8a-bac1-3a7cb8088074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:35.761545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86785fc7-123a-4c8a-bac1-3a7cb8088074 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.761828 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86785fc7-123a-4c8a-bac1-3a7cb8088074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.762108 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86785fc7-123a-4c8a-bac1-3a7cb8088074 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.762758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86785fc7-123a-4c8a-bac1-3a7cb8088074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:35.779732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86785fc7-123a-4c8a-bac1-3a7cb8088074 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:35.791360 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861e0c0-5ffa-41b1-a4cb-649b0fb6981f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:35.793016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861e0c0-5ffa-41b1-a4cb-649b0fb6981f admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.794050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861e0c0-5ffa-41b1-a4cb-649b0fb6981f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.794050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861e0c0-5ffa-41b1-a4cb-649b0fb6981f admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.795725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1056/2097] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:35.798661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861e0c0-5ffa-41b1-a4cb-649b0fb6981f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:35.800080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86785fc7-123a-4c8a-bac1-3a7cb8088074 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:35.803687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861e0c0-5ffa-41b1-a4cb-649b0fb6981f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:35.817914 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86785fc7-123a-4c8a-bac1-3a7cb8088074 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:35.825519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1042/2098] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:35.831019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861e0c0-5ffa-41b1-a4cb-649b0fb6981f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:35.833066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7ec90-d151-4832-be90-57ba731f922f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:35.834759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7ec90-d151-4832-be90-57ba731f922f admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.835238 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7ec90-d151-4832-be90-57ba731f922f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.836476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7ec90-d151-4832-be90-57ba731f922f admin admin] PATH_INFO: `/v3/projects/069464e6363946ef9efc3e1145c524a6/users/59fe590a4b744e1c8270d968773ee77a/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.840987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7ec90-d151-4832-be90-57ba731f922f admin admin] RBAC: Authorizing `identity:create_grant(project_id=069464e6363946ef9efc3e1145c524a6, user_id=59fe590a4b744e1c8270d968773ee77a, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:35.844641 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861e0c0-5ffa-41b1-a4cb-649b0fb6981f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:35.844783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7ec90-d151-4832-be90-57ba731f922f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:35.850982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67576ae-0b52-43a0-b2b1-5a25799e2bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:35.851828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67576ae-0b52-43a0-b2b1-5a25799e2bcd admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.852118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67576ae-0b52-43a0-b2b1-5a25799e2bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.852859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67576ae-0b52-43a0-b2b1-5a25799e2bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.853633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67576ae-0b52-43a0-b2b1-5a25799e2bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:35.855340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67576ae-0b52-43a0-b2b1-5a25799e2bcd admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:35.858092 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1057/2099] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:35 2022] PUT /identity/v3/projects/069464e6363946ef9efc3e1145c524a6/users/59fe590a4b744e1c8270d968773ee77a/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:35.866733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1043/2100] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:35.869691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfbf5f-2c0f-446b-8d9d-78bfc6f17e55 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.870206 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfbf5f-2c0f-446b-8d9d-78bfc6f17e55 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.870643 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfbf5f-2c0f-446b-8d9d-78bfc6f17e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.870961 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67576ae-0b52-43a0-b2b1-5a25799e2bcd admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:35.881427 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67576ae-0b52-43a0-b2b1-5a25799e2bcd admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:35.913470 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-58cfbf5f-2c0f-446b-8d9d-78bfc6f17e55 None None] MFA Rules not processed for user `59fe590a4b744e1c8270d968773ee77a`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:35.916034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-58cfbf5f-2c0f-446b-8d9d-78bfc6f17e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:35.951953 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca859a-f897-47d3-8447-570c96aece61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:35.952524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca859a-f897-47d3-8447-570c96aece61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.952803 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca859a-f897-47d3-8447-570c96aece61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.953080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca859a-f897-47d3-8447-570c96aece61 admin admin] PATH_INFO: `/v3/projects/37c7f2aa956b42ab8d5fc77210dcac3a/users/8bdaada534ec4569bd2cb711e7c722ad/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.968939 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca859a-f897-47d3-8447-570c96aece61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37c7f2aa956b42ab8d5fc77210dcac3a, user_id=8bdaada534ec4569bd2cb711e7c722ad, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:35.972404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca859a-f897-47d3-8447-570c96aece61 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:35.995046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1058/2101] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:35 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:36.005929 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfbf5f-2c0f-446b-8d9d-78bfc6f17e55 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:36.007321 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-58cfbf5f-2c0f-446b-8d9d-78bfc6f17e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:36.059602 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:36.061453 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1044/2102] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:35 2022] PUT /identity/v3/projects/37c7f2aa956b42ab8d5fc77210dcac3a/users/8bdaada534ec4569bd2cb711e7c722ad/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:26:36.072333 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995316b-a4a7-4e14-9b4d-c89ee069ad18 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:36.072768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995316b-a4a7-4e14-9b4d-c89ee069ad18 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:36.073329 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995316b-a4a7-4e14-9b4d-c89ee069ad18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:36.087937 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfbf5f-2c0f-446b-8d9d-78bfc6f17e55 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:36.122498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-9995316b-a4a7-4e14-9b4d-c89ee069ad18 None None] MFA Rules not processed for user `8bdaada534ec4569bd2cb711e7c722ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:36.124266 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-9995316b-a4a7-4e14-9b4d-c89ee069ad18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:36.150187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4e2f6-0245-426f-8916-82bdb44d8be4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:36.150840 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4e2f6-0245-426f-8916-82bdb44d8be4 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:36.151127 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4e2f6-0245-426f-8916-82bdb44d8be4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:36.151492 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4e2f6-0245-426f-8916-82bdb44d8be4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:36.212444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1045/2103] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:36.221223 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995316b-a4a7-4e14-9b4d-c89ee069ad18 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:36.232032 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4e2f6-0245-426f-8916-82bdb44d8be4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:36.233617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4e2f6-0245-426f-8916-82bdb44d8be4 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:36.249220 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1059/2104] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:36 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:26:36.311573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995316b-a4a7-4e14-9b4d-c89ee069ad18 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:36.316452 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cc0c2-19a1-47ea-b14c-831be5ed713f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:36.317076 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cc0c2-19a1-47ea-b14c-831be5ed713f admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:36.317357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cc0c2-19a1-47ea-b14c-831be5ed713f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:36.317628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cc0c2-19a1-47ea-b14c-831be5ed713f admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:36.318261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2cc0c2-19a1-47ea-b14c-831be5ed713f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:36.320207 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2cc0c2-19a1-47ea-b14c-831be5ed713f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:36.465992 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1046/2105] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:36 2022] POST /identity/v3/projects => generated 393 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:36.470046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4e2f6-0245-426f-8916-82bdb44d8be4 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:36.481543 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4e2f6-0245-426f-8916-82bdb44d8be4 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:36.547142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02da09-8b73-4b87-942b-e53f71e12ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:36.547142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02da09-8b73-4b87-942b-e53f71e12ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:36.547142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02da09-8b73-4b87-942b-e53f71e12ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:36.547142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02da09-8b73-4b87-942b-e53f71e12ab6 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:36.548565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02da09-8b73-4b87-942b-e53f71e12ab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:36.548565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02da09-8b73-4b87-942b-e53f71e12ab6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:36.669454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1060/2106] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:36 2022] POST /identity/v3/users => generated 303 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:26:36.673569 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cc0c2-19a1-47ea-b14c-831be5ed713f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:36.684467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cc0c2-19a1-47ea-b14c-831be5ed713f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:36.689071 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d202a23-cb23-4c4f-a41d-f4f1f2bf51e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:36.689658 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d202a23-cb23-4c4f-a41d-f4f1f2bf51e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:36.689944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d202a23-cb23-4c4f-a41d-f4f1f2bf51e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:36.690220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d202a23-cb23-4c4f-a41d-f4f1f2bf51e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:36.690867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d202a23-cb23-4c4f-a41d-f4f1f2bf51e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:36.692534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d202a23-cb23-4c4f-a41d-f4f1f2bf51e8 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:36.703670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1047/2107] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:36.708243 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02da09-8b73-4b87-942b-e53f71e12ab6 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:36.724126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02da09-8b73-4b87-942b-e53f71e12ab6 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:36.731033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d830e-87de-429b-bad8-6b6da773a566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:36.731959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d830e-87de-429b-bad8-6b6da773a566 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:36.732353 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d830e-87de-429b-bad8-6b6da773a566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:36.732825 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d830e-87de-429b-bad8-6b6da773a566 admin admin] PATH_INFO: `/v3/projects/b1b4b5cf35ff448aa395ae69a55a30a5/users/88b2427e633c43ff9cb06b8e124707d6/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:36.752913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d830e-87de-429b-bad8-6b6da773a566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b4b5cf35ff448aa395ae69a55a30a5, user_id=88b2427e633c43ff9cb06b8e124707d6, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:36.756676 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d830e-87de-429b-bad8-6b6da773a566 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:36.803131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d202a23-cb23-4c4f-a41d-f4f1f2bf51e8 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:36.804977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2d202a23-cb23-4c4f-a41d-f4f1f2bf51e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:36.887278 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1061/2108] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:36 2022] PUT /identity/v3/projects/b1b4b5cf35ff448aa395ae69a55a30a5/users/88b2427e633c43ff9cb06b8e124707d6/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:36.889304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d202a23-cb23-4c4f-a41d-f4f1f2bf51e8 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:36.891094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d830e-87de-429b-bad8-6b6da773a566 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:36.946974 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628830aa-f4b8-4e13-afac-42f5fa63a79e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:36.947646 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628830aa-f4b8-4e13-afac-42f5fa63a79e service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:36.947917 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628830aa-f4b8-4e13-afac-42f5fa63a79e service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:36.948225 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628830aa-f4b8-4e13-afac-42f5fa63a79e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:36.950719 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-628830aa-f4b8-4e13-afac-42f5fa63a79e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:36.958163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d830e-87de-429b-bad8-6b6da773a566 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:36.963634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3c181-d2ed-4a24-a20a-dce105e2795f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:36.965297 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3c181-d2ed-4a24-a20a-dce105e2795f admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:36.965588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3c181-d2ed-4a24-a20a-dce105e2795f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:36.965887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3c181-d2ed-4a24-a20a-dce105e2795f admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:36.966649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c3c181-d2ed-4a24-a20a-dce105e2795f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:36.968278 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c3c181-d2ed-4a24-a20a-dce105e2795f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:36.978472 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1062/2109] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:36.983493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3c181-d2ed-4a24-a20a-dce105e2795f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:36.993405 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3c181-d2ed-4a24-a20a-dce105e2795f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:36.998809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76e3d5-6c57-4947-8549-325db6d1a7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:36.999658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76e3d5-6c57-4947-8549-325db6d1a7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:36.999952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76e3d5-6c57-4947-8549-325db6d1a7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:37.000233 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76e3d5-6c57-4947-8549-325db6d1a7f7 admin admin] PATH_INFO: `/v3/projects/b1b4b5cf35ff448aa395ae69a55a30a5/users/88b2427e633c43ff9cb06b8e124707d6/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:37.002314 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76e3d5-6c57-4947-8549-325db6d1a7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b4b5cf35ff448aa395ae69a55a30a5, user_id=88b2427e633c43ff9cb06b8e124707d6, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:37.005064 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76e3d5-6c57-4947-8549-325db6d1a7f7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:37.016418 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1063/2110] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:36 2022] PUT /identity/v3/projects/b1b4b5cf35ff448aa395ae69a55a30a5/users/88b2427e633c43ff9cb06b8e124707d6/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:37.020252 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628830aa-f4b8-4e13-afac-42f5fa63a79e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:37.021679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628830aa-f4b8-4e13-afac-42f5fa63a79e service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:37.028693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae69bda-3b24-4730-8440-4672b4f62031 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:37.028984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae69bda-3b24-4730-8440-4672b4f62031 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:37.029263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae69bda-3b24-4730-8440-4672b4f62031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:37.031965 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1048/2111] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:36 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:37.060366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-5ae69bda-3b24-4730-8440-4672b4f62031 None None] MFA Rules not processed for user `88b2427e633c43ff9cb06b8e124707d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:37.061610 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5ae69bda-3b24-4730-8440-4672b4f62031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:37.091551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:37.142766 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1064/2112] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:37 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:37.625798 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628830aa-f4b8-4e13-afac-42f5fa63a79e service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:37.635789 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628830aa-f4b8-4e13-afac-42f5fa63a79e service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:37.694481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b301e2-798b-4968-9691-ca4303f7df18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:37.695004 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b301e2-798b-4968-9691-ca4303f7df18 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:37.695327 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b301e2-798b-4968-9691-ca4303f7df18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:37.695603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b301e2-798b-4968-9691-ca4303f7df18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:37.705981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b301e2-798b-4968-9691-ca4303f7df18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:37.707501 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b301e2-798b-4968-9691-ca4303f7df18 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:37.717512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1049/2113] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:37 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:37.891448 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae69bda-3b24-4730-8440-4672b4f62031 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:37.903551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae69bda-3b24-4730-8440-4672b4f62031 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:37.907283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2d490-6c61-462a-b44f-32257ae0d2e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:37.919569 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2d490-6c61-462a-b44f-32257ae0d2e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:37.919923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2d490-6c61-462a-b44f-32257ae0d2e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:37.920208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2d490-6c61-462a-b44f-32257ae0d2e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:37.987410 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2d490-6c61-462a-b44f-32257ae0d2e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:37.988831 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2d490-6c61-462a-b44f-32257ae0d2e1 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:37.997989 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1065/2114] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:38.404792 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:38.861814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:39.140805 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b301e2-798b-4968-9691-ca4303f7df18 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:39.151859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-83b301e2-798b-4968-9691-ca4303f7df18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:39.213468 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b301e2-798b-4968-9691-ca4303f7df18 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:39.219234 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd42478-a938-4493-ae87-c9b6d77e817e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:39.220351 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd42478-a938-4493-ae87-c9b6d77e817e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:39.220554 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd42478-a938-4493-ae87-c9b6d77e817e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:39.220773 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd42478-a938-4493-ae87-c9b6d77e817e admin admin] PATH_INFO: `/v3/users/e5abf021321f4c92b9fbe6e64965a6c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:39.222358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd42478-a938-4493-ae87-c9b6d77e817e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5abf021321f4c92b9fbe6e64965a6c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:39.224382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd42478-a938-4493-ae87-c9b6d77e817e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:39.528929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-acd42478-a938-4493-ae87-c9b6d77e817e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5abf021321f4c92b9fbe6e64965a6c1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:39.540842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-acd42478-a938-4493-ae87-c9b6d77e817e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5abf021321f4c92b9fbe6e64965a6c1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:39.548477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-acd42478-a938-4493-ae87-c9b6d77e817e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5abf021321f4c92b9fbe6e64965a6c1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:39.639256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-acd42478-a938-4493-ae87-c9b6d77e817e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5abf021321f4c92b9fbe6e64965a6c1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:39.639879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-acd42478-a938-4493-ae87-c9b6d77e817e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5abf021321f4c92b9fbe6e64965a6c1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:39.646451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1050/2115] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:39 2022] DELETE /identity/v3/users/e5abf021321f4c92b9fbe6e64965a6c1 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:39.651342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d91f4-9d2c-4bb2-b0ae-19994e58dc71 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:39.652450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d91f4-9d2c-4bb2-b0ae-19994e58dc71 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:39.652874 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d91f4-9d2c-4bb2-b0ae-19994e58dc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:39.672032 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a41d91f4-9d2c-4bb2-b0ae-19994e58dc71 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:39.673263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a41d91f4-9d2c-4bb2-b0ae-19994e58dc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:39.733547 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1066/2116] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:39.750015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd42478-a938-4493-ae87-c9b6d77e817e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:39.751502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-acd42478-a938-4493-ae87-c9b6d77e817e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:39.819153 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd42478-a938-4493-ae87-c9b6d77e817e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:39.822802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b62372-0180-43ba-b411-7e66833f5a4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:39.824063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b62372-0180-43ba-b411-7e66833f5a4f service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:39.824400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b62372-0180-43ba-b411-7e66833f5a4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:39.824761 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b62372-0180-43ba-b411-7e66833f5a4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:39.860555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b62372-0180-43ba-b411-7e66833f5a4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:39.864520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b62372-0180-43ba-b411-7e66833f5a4f service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:39.886585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1051/2117] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:26:40.121743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:40.262407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:40.326436 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d91f4-9d2c-4bb2-b0ae-19994e58dc71 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:40.327948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a41d91f4-9d2c-4bb2-b0ae-19994e58dc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:40.342215 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d91f4-9d2c-4bb2-b0ae-19994e58dc71 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:40.347028 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e69bee-46e4-419b-a88b-1d8351e728ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:40.347707 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e69bee-46e4-419b-a88b-1d8351e728ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:40.348034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e69bee-46e4-419b-a88b-1d8351e728ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:40.348273 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e69bee-46e4-419b-a88b-1d8351e728ca admin admin] PATH_INFO: `/v3/projects/07d188f29f7948b2a7f3333b7a60bddc` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:40.351110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e69bee-46e4-419b-a88b-1d8351e728ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=07d188f29f7948b2a7f3333b7a60bddc)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:40.351508 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e69bee-46e4-419b-a88b-1d8351e728ca admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:40.551479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-70e69bee-46e4-419b-a88b-1d8351e728ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07d188f29f7948b2a7f3333b7a60bddc', 'request_id': 'req-70e69bee-46e4-419b-a88b-1d8351e728ca'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:40.650956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-70e69bee-46e4-419b-a88b-1d8351e728ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07d188f29f7948b2a7f3333b7a60bddc', 'request_id': 'req-70e69bee-46e4-419b-a88b-1d8351e728ca'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:40.660216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-70e69bee-46e4-419b-a88b-1d8351e728ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07d188f29f7948b2a7f3333b7a60bddc', 'request_id': 'req-70e69bee-46e4-419b-a88b-1d8351e728ca'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:40.671725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-70e69bee-46e4-419b-a88b-1d8351e728ca admin admin] The token cache is being invalidate because project 07d188f29f7948b2a7f3333b7a60bddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:40.672020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-70e69bee-46e4-419b-a88b-1d8351e728ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:40.672724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-70e69bee-46e4-419b-a88b-1d8351e728ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:40.674249 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1067/2118] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:40 2022] DELETE /identity/v3/projects/07d188f29f7948b2a7f3333b7a60bddc => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:40.679472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb557685-db74-46ae-955e-9ab011a7703a None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:40.679759 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb557685-db74-46ae-955e-9ab011a7703a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:40.680034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb557685-db74-46ae-955e-9ab011a7703a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:40.702064 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-fb557685-db74-46ae-955e-9ab011a7703a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:40.703350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-fb557685-db74-46ae-955e-9ab011a7703a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:40.772088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1052/2119] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:40.781038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae302e77-2575-4c3b-8519-6faacd25ce63 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:40.781440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae302e77-2575-4c3b-8519-6faacd25ce63 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:40.781820 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae302e77-2575-4c3b-8519-6faacd25ce63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:40.803033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ae302e77-2575-4c3b-8519-6faacd25ce63 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:40.804532 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ae302e77-2575-4c3b-8519-6faacd25ce63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:40.811135 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:40.815642 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1068/2120] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:40.819761 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb557685-db74-46ae-955e-9ab011a7703a None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:40.830599 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb557685-db74-46ae-955e-9ab011a7703a None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:40.835978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bd09b-022f-45fd-b372-6835ad4b954b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:40.836669 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bd09b-022f-45fd-b372-6835ad4b954b admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:40.837046 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bd09b-022f-45fd-b372-6835ad4b954b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:40.837382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bd09b-022f-45fd-b372-6835ad4b954b admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:40.838105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bd09b-022f-45fd-b372-6835ad4b954b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:26:40.838478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bd09b-022f-45fd-b372-6835ad4b954b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:40.840515 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bd09b-022f-45fd-b372-6835ad4b954b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:40.853264 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1053/2121] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:26:40.855926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:40.857469 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae302e77-2575-4c3b-8519-6faacd25ce63 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:40.868005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae302e77-2575-4c3b-8519-6faacd25ce63 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:40.873640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc20d3-1b2a-4962-b41e-1aeaa4bf6d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:40.874441 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc20d3-1b2a-4962-b41e-1aeaa4bf6d4d admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:40.875712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc20d3-1b2a-4962-b41e-1aeaa4bf6d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:40.876165 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc20d3-1b2a-4962-b41e-1aeaa4bf6d4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:40.876959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc20d3-1b2a-4962-b41e-1aeaa4bf6d4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:40.880172 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc20d3-1b2a-4962-b41e-1aeaa4bf6d4d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:40.992799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1069/2122] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:40 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:40.997328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bd09b-022f-45fd-b372-6835ad4b954b admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:41.007699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bd09b-022f-45fd-b372-6835ad4b954b admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:41.056656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:41.080976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7c303-cce3-42c5-9c66-26d9a154c9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:41.081786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7c303-cce3-42c5-9c66-26d9a154c9ae admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:41.082164 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7c303-cce3-42c5-9c66-26d9a154c9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:41.082538 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7c303-cce3-42c5-9c66-26d9a154c9ae admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:41.083430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7c303-cce3-42c5-9c66-26d9a154c9ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:41.085563 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7c303-cce3-42c5-9c66-26d9a154c9ae admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:41.183584 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1054/2123] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:40 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:41.187228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc20d3-1b2a-4962-b41e-1aeaa4bf6d4d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:41.198437 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc20d3-1b2a-4962-b41e-1aeaa4bf6d4d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:41.207351 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae8f0b1-651e-45ef-bacb-0c8cf19f3332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:41.219231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8f0b1-651e-45ef-bacb-0c8cf19f3332 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:41.219231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8f0b1-651e-45ef-bacb-0c8cf19f3332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:41.219231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8f0b1-651e-45ef-bacb-0c8cf19f3332 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:41.219231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae8f0b1-651e-45ef-bacb-0c8cf19f3332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:41.219231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae8f0b1-651e-45ef-bacb-0c8cf19f3332 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:41.232222 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1070/2124] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:41.235952 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7c303-cce3-42c5-9c66-26d9a154c9ae admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:41.269396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7c303-cce3-42c5-9c66-26d9a154c9ae admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:41.275453 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9bcd05-4517-4590-ab13-9e747dd6805b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:41.276204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bcd05-4517-4590-ab13-9e747dd6805b admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:41.276678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bcd05-4517-4590-ab13-9e747dd6805b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:41.277047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bcd05-4517-4590-ab13-9e747dd6805b admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:41.277796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9bcd05-4517-4590-ab13-9e747dd6805b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:41.279720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9bcd05-4517-4590-ab13-9e747dd6805b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:41.293957 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1055/2125] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:26:41.298013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae8f0b1-651e-45ef-bacb-0c8cf19f3332 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:41.326408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae8f0b1-651e-45ef-bacb-0c8cf19f3332 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:41.331610 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f387f-7b1f-471b-9086-f94794e76368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:41.332290 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f387f-7b1f-471b-9086-f94794e76368 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:41.332632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f387f-7b1f-471b-9086-f94794e76368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:41.336844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f387f-7b1f-471b-9086-f94794e76368 admin admin] PATH_INFO: `/v3/projects/ec8ddabe0fd049de97da2f63453aabef/users/28638075e935490bb7c8c229de9d14c5/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:41.348807 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f387f-7b1f-471b-9086-f94794e76368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8ddabe0fd049de97da2f63453aabef, user_id=28638075e935490bb7c8c229de9d14c5, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:41.351906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f387f-7b1f-471b-9086-f94794e76368 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:41.359510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:41.444886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1071/2126] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:41 2022] PUT /identity/v3/projects/ec8ddabe0fd049de97da2f63453aabef/users/28638075e935490bb7c8c229de9d14c5/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:41.450319 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb750d-c27c-46b5-9ef3-176b3a049a2c None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:41.450677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb750d-c27c-46b5-9ef3-176b3a049a2c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:41.451180 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb750d-c27c-46b5-9ef3-176b3a049a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:41.473452 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-5feb750d-c27c-46b5-9ef3-176b3a049a2c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:41.474723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5feb750d-c27c-46b5-9ef3-176b3a049a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:41.550867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1056/2127] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:41.559705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f387f-7b1f-471b-9086-f94794e76368 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:41.560056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-607f387f-7b1f-471b-9086-f94794e76368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:41.628226 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f387f-7b1f-471b-9086-f94794e76368 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:41.632017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1e743-3572-4236-9501-1cc535c95d1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:41.632657 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1e743-3572-4236-9501-1cc535c95d1a service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:41.632940 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1e743-3572-4236-9501-1cc535c95d1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:41.633211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1e743-3572-4236-9501-1cc535c95d1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:41.643372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1e743-3572-4236-9501-1cc535c95d1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:41.644836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1e743-3572-4236-9501-1cc535c95d1a service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:41.663421 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1072/2128] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:44.971383 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb750d-c27c-46b5-9ef3-176b3a049a2c None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:44.974512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5feb750d-c27c-46b5-9ef3-176b3a049a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:45.006596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb750d-c27c-46b5-9ef3-176b3a049a2c None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:45.020934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bb0a2-0878-4640-9e9e-5b05407ca364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:45.020934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bb0a2-0878-4640-9e9e-5b05407ca364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:45.020934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bb0a2-0878-4640-9e9e-5b05407ca364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:45.020934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bb0a2-0878-4640-9e9e-5b05407ca364 admin admin] PATH_INFO: `/v3/users/8bdaada534ec4569bd2cb711e7c722ad` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:45.021330 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bb0a2-0878-4640-9e9e-5b05407ca364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bdaada534ec4569bd2cb711e7c722ad)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:45.021330 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bb0a2-0878-4640-9e9e-5b05407ca364 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:45.406439 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-280bb0a2-0878-4640-9e9e-5b05407ca364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bdaada534ec4569bd2cb711e7c722ad'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:45.410413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:45.434930 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-280bb0a2-0878-4640-9e9e-5b05407ca364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bdaada534ec4569bd2cb711e7c722ad'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:45.455812 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-280bb0a2-0878-4640-9e9e-5b05407ca364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bdaada534ec4569bd2cb711e7c722ad'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:45.533462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-280bb0a2-0878-4640-9e9e-5b05407ca364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bdaada534ec4569bd2cb711e7c722ad'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:45.534153 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-280bb0a2-0878-4640-9e9e-5b05407ca364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bdaada534ec4569bd2cb711e7c722ad'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:45.570052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1057/2129] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:44 2022] DELETE /identity/v3/users/8bdaada534ec4569bd2cb711e7c722ad => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:45.574929 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d68ce-89ee-4a09-807e-585abd576735 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:45.576990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d68ce-89ee-4a09-807e-585abd576735 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:45.578657 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d68ce-89ee-4a09-807e-585abd576735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:45.615058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-216d68ce-89ee-4a09-807e-585abd576735 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:45.616571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-216d68ce-89ee-4a09-807e-585abd576735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:45.735639 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1073/2130] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:45.746896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bb0a2-0878-4640-9e9e-5b05407ca364 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:45.749107 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-280bb0a2-0878-4640-9e9e-5b05407ca364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:45.831346 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bb0a2-0878-4640-9e9e-5b05407ca364 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:45.835406 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfbb59-f677-421d-9978-baac88afa0ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:45.836031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfbb59-f677-421d-9978-baac88afa0ea service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:45.836319 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfbb59-f677-421d-9978-baac88afa0ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:45.836593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfbb59-f677-421d-9978-baac88afa0ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:45.847432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfbb59-f677-421d-9978-baac88afa0ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:45.849220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfbb59-f677-421d-9978-baac88afa0ea service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:45.861549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1058/2131] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:46.217446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e02051-5680-4f8e-b27a-74005a9d3e19 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:46.220836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e02051-5680-4f8e-b27a-74005a9d3e19 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:46.221692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e02051-5680-4f8e-b27a-74005a9d3e19 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:46.224688 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1074/2132] 10.222.0.78 () {54 vars in 921 bytes} [Mon Sep 19 11:26:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:26:46.240138 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c0929-cd1f-4b70-84ba-cd27fedcc67d None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:46.240663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c0929-cd1f-4b70-84ba-cd27fedcc67d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:46.240663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c0929-cd1f-4b70-84ba-cd27fedcc67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:46.280784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-989c0929-cd1f-4b70-84ba-cd27fedcc67d None None] MFA Rules not processed for user `0427cdd3905a422cab4fed0aac187ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:46.282288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-989c0929-cd1f-4b70-84ba-cd27fedcc67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:46.402436 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e02051-5680-4f8e-b27a-74005a9d3e19 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:46.404884 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d1e02051-5680-4f8e-b27a-74005a9d3e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:46.457261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1059/2133] 10.222.0.78 () {58 vars in 1018 bytes} [Mon Sep 19 11:26:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:46.480760 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e02051-5680-4f8e-b27a-74005a9d3e19 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:46.484632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c0929-cd1f-4b70-84ba-cd27fedcc67d None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:46.486126 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-989c0929-cd1f-4b70-84ba-cd27fedcc67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:46.500143 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2c2ad-d61b-4b8f-a793-590395092b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:46.502740 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2c2ad-d61b-4b8f-a793-590395092b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:46.503241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2c2ad-d61b-4b8f-a793-590395092b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:46.504700 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2c2ad-d61b-4b8f-a793-590395092b97 admin admin] PATH_INFO: `/v3/projects/37c7f2aa956b42ab8d5fc77210dcac3a` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:46.508187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2c2ad-d61b-4b8f-a793-590395092b97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37c7f2aa956b42ab8d5fc77210dcac3a)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:46.519099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c0929-cd1f-4b70-84ba-cd27fedcc67d None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:46.522615 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2c2ad-d61b-4b8f-a793-590395092b97 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:46.530426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49f2b8-77b3-4b04-a317-b2f16d42939b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:46.531063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49f2b8-77b3-4b04-a317-b2f16d42939b service placement] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:46.533359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49f2b8-77b3-4b04-a317-b2f16d42939b service placement] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:46.533713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49f2b8-77b3-4b04-a317-b2f16d42939b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:46.553321 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49f2b8-77b3-4b04-a317-b2f16d42939b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:46.553321 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49f2b8-77b3-4b04-a317-b2f16d42939b service placement] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:46.578490 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1060/2134] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:26:46.589983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49f2b8-77b3-4b04-a317-b2f16d42939b service placement] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:46.592709 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-9c49f2b8-77b3-4b04-a317-b2f16d42939b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:46.611661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49f2b8-77b3-4b04-a317-b2f16d42939b service placement] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:46.617679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e751d3-767b-4504-ad8e-97b8a2b62cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:46.618664 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e751d3-767b-4504-ad8e-97b8a2b62cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:46.618958 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e751d3-767b-4504-ad8e-97b8a2b62cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:46.619318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e751d3-767b-4504-ad8e-97b8a2b62cd8 admin admin] PATH_INFO: `/v3/users/eec3ef0842b948deb7ecc46f7c56dd04` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:46.652906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e751d3-767b-4504-ad8e-97b8a2b62cd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eec3ef0842b948deb7ecc46f7c56dd04)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:46.654972 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e751d3-767b-4504-ad8e-97b8a2b62cd8 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:46.777682 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c4f2c2ad-d61b-4b8f-a793-590395092b97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37c7f2aa956b42ab8d5fc77210dcac3a', 'request_id': 'req-c4f2c2ad-d61b-4b8f-a793-590395092b97'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:46.952717 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c4f2c2ad-d61b-4b8f-a793-590395092b97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37c7f2aa956b42ab8d5fc77210dcac3a', 'request_id': 'req-c4f2c2ad-d61b-4b8f-a793-590395092b97'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:46.960873 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c4f2c2ad-d61b-4b8f-a793-590395092b97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37c7f2aa956b42ab8d5fc77210dcac3a', 'request_id': 'req-c4f2c2ad-d61b-4b8f-a793-590395092b97'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:46.972712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c4f2c2ad-d61b-4b8f-a793-590395092b97 admin admin] The token cache is being invalidate because project 37c7f2aa956b42ab8d5fc77210dcac3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:46.973017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c4f2c2ad-d61b-4b8f-a793-590395092b97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:46.973698 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c4f2c2ad-d61b-4b8f-a793-590395092b97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:46.975846 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1075/2135] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:46 2022] DELETE /identity/v3/projects/37c7f2aa956b42ab8d5fc77210dcac3a => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:46.980759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2c2ad-d61b-4b8f-a793-590395092b97 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:46.982047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c4f2c2ad-d61b-4b8f-a793-590395092b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:47.013492 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:47.042669 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-04e751d3-767b-4504-ad8e-97b8a2b62cd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eec3ef0842b948deb7ecc46f7c56dd04'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:47.061709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2c2ad-d61b-4b8f-a793-590395092b97 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:47.064127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-04e751d3-767b-4504-ad8e-97b8a2b62cd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eec3ef0842b948deb7ecc46f7c56dd04'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:47.075084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba86b2-9347-464c-8556-97d94365a174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:47.075084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba86b2-9347-464c-8556-97d94365a174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:47.079463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba86b2-9347-464c-8556-97d94365a174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:47.079463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba86b2-9347-464c-8556-97d94365a174 admin admin] PATH_INFO: `/v3/users/88b2427e633c43ff9cb06b8e124707d6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:47.079463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba86b2-9347-464c-8556-97d94365a174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88b2427e633c43ff9cb06b8e124707d6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:47.079463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba86b2-9347-464c-8556-97d94365a174 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:47.081412 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-04e751d3-767b-4504-ad8e-97b8a2b62cd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eec3ef0842b948deb7ecc46f7c56dd04'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:47.171378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-04e751d3-767b-4504-ad8e-97b8a2b62cd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eec3ef0842b948deb7ecc46f7c56dd04'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:47.172196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-04e751d3-767b-4504-ad8e-97b8a2b62cd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eec3ef0842b948deb7ecc46f7c56dd04'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:47.179593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1061/2136] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:46 2022] DELETE /identity/v3/users/eec3ef0842b948deb7ecc46f7c56dd04 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:47.347447 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-01ba86b2-9347-464c-8556-97d94365a174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88b2427e633c43ff9cb06b8e124707d6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:47.363512 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-01ba86b2-9347-464c-8556-97d94365a174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88b2427e633c43ff9cb06b8e124707d6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:47.376638 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-01ba86b2-9347-464c-8556-97d94365a174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88b2427e633c43ff9cb06b8e124707d6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:47.525561 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e751d3-767b-4504-ad8e-97b8a2b62cd8 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:47.527040 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-04e751d3-767b-4504-ad8e-97b8a2b62cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:47.552892 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-01ba86b2-9347-464c-8556-97d94365a174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88b2427e633c43ff9cb06b8e124707d6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:47.561365 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-01ba86b2-9347-464c-8556-97d94365a174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88b2427e633c43ff9cb06b8e124707d6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:47.565008 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1076/2137] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:46 2022] DELETE /identity/v3/users/88b2427e633c43ff9cb06b8e124707d6 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:26:47.602355 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e751d3-767b-4504-ad8e-97b8a2b62cd8 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:47.666512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba933fff-a256-4a70-af53-47fcd64de3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:47.667441 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba933fff-a256-4a70-af53-47fcd64de3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:47.667710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba933fff-a256-4a70-af53-47fcd64de3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:47.667981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba933fff-a256-4a70-af53-47fcd64de3e3 admin admin] PATH_INFO: `/v3/projects/089d6ade5a3d4e48868bc6cc190c6251` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:47.669171 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba933fff-a256-4a70-af53-47fcd64de3e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=089d6ade5a3d4e48868bc6cc190c6251)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:47.671141 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba933fff-a256-4a70-af53-47fcd64de3e3 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:47.697689 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:47.771827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65909f-1425-49da-8cbc-b3730ef6ced8 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:47.772119 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65909f-1425-49da-8cbc-b3730ef6ced8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:47.772570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65909f-1425-49da-8cbc-b3730ef6ced8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:47.800648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-dd65909f-1425-49da-8cbc-b3730ef6ced8 None None] MFA Rules not processed for user `e81d8e2a42f34cc0b4f3ca54eb203f59`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:47.801960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-dd65909f-1425-49da-8cbc-b3730ef6ced8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:47.852348 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ba933fff-a256-4a70-af53-47fcd64de3e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '089d6ade5a3d4e48868bc6cc190c6251', 'request_id': 'req-ba933fff-a256-4a70-af53-47fcd64de3e3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:47.867706 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ba933fff-a256-4a70-af53-47fcd64de3e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '089d6ade5a3d4e48868bc6cc190c6251', 'request_id': 'req-ba933fff-a256-4a70-af53-47fcd64de3e3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:47.894430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1077/2138] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:47.976149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ba933fff-a256-4a70-af53-47fcd64de3e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '089d6ade5a3d4e48868bc6cc190c6251', 'request_id': 'req-ba933fff-a256-4a70-af53-47fcd64de3e3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:48.000876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ba933fff-a256-4a70-af53-47fcd64de3e3 admin admin] The token cache is being invalidate because project 089d6ade5a3d4e48868bc6cc190c6251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:48.001176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ba933fff-a256-4a70-af53-47fcd64de3e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:48.001880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ba933fff-a256-4a70-af53-47fcd64de3e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:48.013880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1062/2139] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:47 2022] DELETE /identity/v3/projects/089d6ade5a3d4e48868bc6cc190c6251 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:26:48.019059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57879d56-47f3-4c5c-a44f-ee015b90a3f5 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:48.019691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57879d56-47f3-4c5c-a44f-ee015b90a3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:48.020201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57879d56-47f3-4c5c-a44f-ee015b90a3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:48.043888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-57879d56-47f3-4c5c-a44f-ee015b90a3f5 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:48.045524 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-57879d56-47f3-4c5c-a44f-ee015b90a3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:48.137653 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1078/2140] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:48.145357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60527a38-5f3b-4fca-9ed0-26d8f1063d3d None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:48.145647 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60527a38-5f3b-4fca-9ed0-26d8f1063d3d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:48.145931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60527a38-5f3b-4fca-9ed0-26d8f1063d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:48.169438 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-60527a38-5f3b-4fca-9ed0-26d8f1063d3d None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:48.176204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-60527a38-5f3b-4fca-9ed0-26d8f1063d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:48.190888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57879d56-47f3-4c5c-a44f-ee015b90a3f5 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:48.197902 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1063/2141] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:48.201491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57879d56-47f3-4c5c-a44f-ee015b90a3f5 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:48.210942 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c889efa-6511-4558-9950-6e0f607e4e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:48.212246 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c889efa-6511-4558-9950-6e0f607e4e29 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:48.212539 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c889efa-6511-4558-9950-6e0f607e4e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:48.212819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c889efa-6511-4558-9950-6e0f607e4e29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:48.214561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c889efa-6511-4558-9950-6e0f607e4e29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:26:48.214561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c889efa-6511-4558-9950-6e0f607e4e29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:48.217416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c889efa-6511-4558-9950-6e0f607e4e29 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:48.240503 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60527a38-5f3b-4fca-9ed0-26d8f1063d3d None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:48.246503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1079/2142] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:26:48.251845 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60527a38-5f3b-4fca-9ed0-26d8f1063d3d None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:48.257920 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a732e-b856-44a8-964d-907d0c66afff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:48.258684 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a732e-b856-44a8-964d-907d0c66afff admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:48.259058 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a732e-b856-44a8-964d-907d0c66afff admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:48.259744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a732e-b856-44a8-964d-907d0c66afff admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:48.261900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a732e-b856-44a8-964d-907d0c66afff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:48.269097 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a732e-b856-44a8-964d-907d0c66afff admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:48.302024 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c889efa-6511-4558-9950-6e0f607e4e29 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:48.305843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6c889efa-6511-4558-9950-6e0f607e4e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:48.322743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c889efa-6511-4558-9950-6e0f607e4e29 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:48.333774 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac108f8-df10-4ca6-8083-fbe9ca0d14a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:48.334766 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac108f8-df10-4ca6-8083-fbe9ca0d14a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:48.335218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac108f8-df10-4ca6-8083-fbe9ca0d14a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:48.335634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac108f8-df10-4ca6-8083-fbe9ca0d14a6 admin admin] PATH_INFO: `/v3/projects/b1b4b5cf35ff448aa395ae69a55a30a5` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:48.336958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac108f8-df10-4ca6-8083-fbe9ca0d14a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1b4b5cf35ff448aa395ae69a55a30a5)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:48.340742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac108f8-df10-4ca6-8083-fbe9ca0d14a6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:48.528043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1064/2143] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:48 2022] POST /identity/v3/projects => generated 389 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:26:48.529612 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a732e-b856-44a8-964d-907d0c66afff admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:48.545490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4ac108f8-df10-4ca6-8083-fbe9ca0d14a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1b4b5cf35ff448aa395ae69a55a30a5', 'request_id': 'req-4ac108f8-df10-4ca6-8083-fbe9ca0d14a6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:48.610919 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a732e-b856-44a8-964d-907d0c66afff admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:48.630781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94c7d4-6670-453e-9fe0-44eab9028f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:48.631459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94c7d4-6670-453e-9fe0-44eab9028f76 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:48.631747 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94c7d4-6670-453e-9fe0-44eab9028f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:48.632028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94c7d4-6670-453e-9fe0-44eab9028f76 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:48.632675 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94c7d4-6670-453e-9fe0-44eab9028f76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:48.636103 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94c7d4-6670-453e-9fe0-44eab9028f76 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:48.638172 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4ac108f8-df10-4ca6-8083-fbe9ca0d14a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1b4b5cf35ff448aa395ae69a55a30a5', 'request_id': 'req-4ac108f8-df10-4ca6-8083-fbe9ca0d14a6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:48.647364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4ac108f8-df10-4ca6-8083-fbe9ca0d14a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1b4b5cf35ff448aa395ae69a55a30a5', 'request_id': 'req-4ac108f8-df10-4ca6-8083-fbe9ca0d14a6'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:48.666841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4ac108f8-df10-4ca6-8083-fbe9ca0d14a6 admin admin] The token cache is being invalidate because project b1b4b5cf35ff448aa395ae69a55a30a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:48.668212 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4ac108f8-df10-4ca6-8083-fbe9ca0d14a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:48.669031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4ac108f8-df10-4ca6-8083-fbe9ca0d14a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:48.671316 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1080/2144] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:48 2022] DELETE /identity/v3/projects/b1b4b5cf35ff448aa395ae69a55a30a5 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:48.676707 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3d4ae-e7e6-44d0-88ea-15023179f135 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:48.677123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3d4ae-e7e6-44d0-88ea-15023179f135 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:48.677756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3d4ae-e7e6-44d0-88ea-15023179f135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:48.702750 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-bbe3d4ae-e7e6-44d0-88ea-15023179f135 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:48.704990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bbe3d4ae-e7e6-44d0-88ea-15023179f135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:48.726640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1081/2145] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:48.726640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3d4ae-e7e6-44d0-88ea-15023179f135 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:48.729728 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bbe3d4ae-e7e6-44d0-88ea-15023179f135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:48.744003 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3d4ae-e7e6-44d0-88ea-15023179f135 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:48.750472 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb8345-6276-42c4-abb5-29dd0257b320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:48.751776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb8345-6276-42c4-abb5-29dd0257b320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:48.752163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb8345-6276-42c4-abb5-29dd0257b320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:48.752546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb8345-6276-42c4-abb5-29dd0257b320 admin admin] PATH_INFO: `/v3/users/ad171360ac174f0f9fe3f7f6579d0836` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:48.754458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb8345-6276-42c4-abb5-29dd0257b320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad171360ac174f0f9fe3f7f6579d0836)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:48.757066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb8345-6276-42c4-abb5-29dd0257b320 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:48.761954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1065/2146] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:48 2022] POST /identity/v3/users => generated 295 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:48.764249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86077cd7-27ae-4158-a975-72e2d57057f6 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:48.765335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86077cd7-27ae-4158-a975-72e2d57057f6 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:48.765778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86077cd7-27ae-4158-a975-72e2d57057f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:48.798029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-86077cd7-27ae-4158-a975-72e2d57057f6 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:48.799320 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-86077cd7-27ae-4158-a975-72e2d57057f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:48.807780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1066/2147] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:48.811835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86077cd7-27ae-4158-a975-72e2d57057f6 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:48.813272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-86077cd7-27ae-4158-a975-72e2d57057f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:48.826401 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86077cd7-27ae-4158-a975-72e2d57057f6 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:48.831253 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7f22b-93aa-4e18-b1d3-054945077a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:48.831934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7f22b-93aa-4e18-b1d3-054945077a99 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:48.832286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7f22b-93aa-4e18-b1d3-054945077a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:48.832627 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7f22b-93aa-4e18-b1d3-054945077a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:48.833343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7f22b-93aa-4e18-b1d3-054945077a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:48.835039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7f22b-93aa-4e18-b1d3-054945077a99 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:48.846592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1067/2148] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:48.847454 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7f22b-93aa-4e18-b1d3-054945077a99 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:48.857308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7f22b-93aa-4e18-b1d3-054945077a99 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:48.862168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82950a-345e-4a60-bfdd-ab4215668b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:48.862781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82950a-345e-4a60-bfdd-ab4215668b5a admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:48.863057 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82950a-345e-4a60-bfdd-ab4215668b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:48.863357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82950a-345e-4a60-bfdd-ab4215668b5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:48.864025 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82950a-345e-4a60-bfdd-ab4215668b5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:26:48.864295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82950a-345e-4a60-bfdd-ab4215668b5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:48.878820 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82950a-345e-4a60-bfdd-ab4215668b5a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:48.879711 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1068/2149] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:26:48.879878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82950a-345e-4a60-bfdd-ab4215668b5a admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:48.891189 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82950a-345e-4a60-bfdd-ab4215668b5a admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:48.898088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808ad0-2f76-49e2-9787-984dadb6c96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:48.898717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808ad0-2f76-49e2-9787-984dadb6c96a admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:48.899001 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808ad0-2f76-49e2-9787-984dadb6c96a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:48.899549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808ad0-2f76-49e2-9787-984dadb6c96a admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:48.900207 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808ad0-2f76-49e2-9787-984dadb6c96a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:48.902553 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808ad0-2f76-49e2-9787-984dadb6c96a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:48.916400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1069/2150] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:48.916400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808ad0-2f76-49e2-9787-984dadb6c96a admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:48.924735 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808ad0-2f76-49e2-9787-984dadb6c96a admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:48.930191 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf291f-2587-409f-bfb8-f888729ac32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:48.930887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf291f-2587-409f-bfb8-f888729ac32e admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:48.931280 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf291f-2587-409f-bfb8-f888729ac32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:48.931619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf291f-2587-409f-bfb8-f888729ac32e admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:48.932822 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf291f-2587-409f-bfb8-f888729ac32e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:48.935324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf291f-2587-409f-bfb8-f888729ac32e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:49.090076 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1070/2151] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:48 2022] POST /identity/v3/projects => generated 399 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:49.090948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf291f-2587-409f-bfb8-f888729ac32e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:49.129916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-19fb8345-6276-42c4-abb5-29dd0257b320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad171360ac174f0f9fe3f7f6579d0836'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:49.158792 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-19fb8345-6276-42c4-abb5-29dd0257b320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad171360ac174f0f9fe3f7f6579d0836'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:49.167114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-19fb8345-6276-42c4-abb5-29dd0257b320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad171360ac174f0f9fe3f7f6579d0836'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:49.170058 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf291f-2587-409f-bfb8-f888729ac32e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:49.176686 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5cef3a-4d7e-44a5-abcb-3d45324f32df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:49.178039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cef3a-4d7e-44a5-abcb-3d45324f32df admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:49.178323 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cef3a-4d7e-44a5-abcb-3d45324f32df admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:49.178732 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cef3a-4d7e-44a5-abcb-3d45324f32df admin admin] PATH_INFO: `/v3/projects/0ddce6e2bc7841d495afa07655d7d25f/users/5a8fb193bdd1448eab774bd1eef437c9/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:49.197794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5cef3a-4d7e-44a5-abcb-3d45324f32df admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ddce6e2bc7841d495afa07655d7d25f, user_id=5a8fb193bdd1448eab774bd1eef437c9, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:49.201728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5cef3a-4d7e-44a5-abcb-3d45324f32df admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:49.278835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-19fb8345-6276-42c4-abb5-29dd0257b320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad171360ac174f0f9fe3f7f6579d0836'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:49.279881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-19fb8345-6276-42c4-abb5-29dd0257b320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad171360ac174f0f9fe3f7f6579d0836'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:49.287998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1082/2152] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:48 2022] DELETE /identity/v3/users/ad171360ac174f0f9fe3f7f6579d0836 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:49.296296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb8345-6276-42c4-abb5-29dd0257b320 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:49.296296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-19fb8345-6276-42c4-abb5-29dd0257b320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:49.360786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1071/2153] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:49 2022] PUT /identity/v3/projects/0ddce6e2bc7841d495afa07655d7d25f/users/5a8fb193bdd1448eab774bd1eef437c9/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:49.361986 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e771-e813-4f75-abc9-7e0668e621fe None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:49.362486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e771-e813-4f75-abc9-7e0668e621fe None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:49.362823 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e771-e813-4f75-abc9-7e0668e621fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:49.415628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-b9a0e771-e813-4f75-abc9-7e0668e621fe None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:49.422937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b9a0e771-e813-4f75-abc9-7e0668e621fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:49.427539 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb8345-6276-42c4-abb5-29dd0257b320 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:49.512962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00d5b4-21b0-427d-95d5-3296c2aaa698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:49.513610 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00d5b4-21b0-427d-95d5-3296c2aaa698 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:49.513896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00d5b4-21b0-427d-95d5-3296c2aaa698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:49.514171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00d5b4-21b0-427d-95d5-3296c2aaa698 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:49.514821 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00d5b4-21b0-427d-95d5-3296c2aaa698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:49.518983 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00d5b4-21b0-427d-95d5-3296c2aaa698 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:49.536853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1072/2154] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:49.541045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616fe82-ab34-46f2-a1e7-3eaf20064b72 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:49.541045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616fe82-ab34-46f2-a1e7-3eaf20064b72 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:49.541045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616fe82-ab34-46f2-a1e7-3eaf20064b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:49.563622 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-6616fe82-ab34-46f2-a1e7-3eaf20064b72 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:49.565002 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-6616fe82-ab34-46f2-a1e7-3eaf20064b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:49.574032 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:49.577333 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1073/2155] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:49.578561 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616fe82-ab34-46f2-a1e7-3eaf20064b72 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:49.580293 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-6616fe82-ab34-46f2-a1e7-3eaf20064b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:49.649706 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616fe82-ab34-46f2-a1e7-3eaf20064b72 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:49.654598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b6f94-dbea-4fca-84c1-a473b180cb05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:49.655251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b6f94-dbea-4fca-84c1-a473b180cb05 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:49.655673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b6f94-dbea-4fca-84c1-a473b180cb05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:49.655834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b6f94-dbea-4fca-84c1-a473b180cb05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:49.669135 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b6f94-dbea-4fca-84c1-a473b180cb05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:49.671403 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b6f94-dbea-4fca-84c1-a473b180cb05 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:49.687237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1074/2156] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:49.688015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b6f94-dbea-4fca-84c1-a473b180cb05 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:49.697417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1083/2157] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:49 2022] POST /identity/v3/users => generated 300 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:49.698145 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-460b6f94-dbea-4fca-84c1-a473b180cb05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:49.701295 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00d5b4-21b0-427d-95d5-3296c2aaa698 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:49.719784 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00d5b4-21b0-427d-95d5-3296c2aaa698 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:49.721007 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b6f94-dbea-4fca-84c1-a473b180cb05 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:49.728739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca66b7a-a817-42ef-a65c-282428fa3950 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:49.730364 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca66b7a-a817-42ef-a65c-282428fa3950 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:49.730980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca66b7a-a817-42ef-a65c-282428fa3950 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:49.731301 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca66b7a-a817-42ef-a65c-282428fa3950 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:49.738218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f685f-fe18-4444-9c90-a6a1ce15dbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:49.738989 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f685f-fe18-4444-9c90-a6a1ce15dbad admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:49.741640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f685f-fe18-4444-9c90-a6a1ce15dbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:49.741640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f685f-fe18-4444-9c90-a6a1ce15dbad admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:49.741640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f685f-fe18-4444-9c90-a6a1ce15dbad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:49.744056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f685f-fe18-4444-9c90-a6a1ce15dbad admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:49.744744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca66b7a-a817-42ef-a65c-282428fa3950 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:49.747741 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca66b7a-a817-42ef-a65c-282428fa3950 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:49.763132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1075/2158] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:49.775228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1084/2159] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:49.778176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca66b7a-a817-42ef-a65c-282428fa3950 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:49.799412 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca66b7a-a817-42ef-a65c-282428fa3950 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:49.823582 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56e116-01c2-4232-a98b-0515a64223d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:49.826494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56e116-01c2-4232-a98b-0515a64223d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:49.826780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56e116-01c2-4232-a98b-0515a64223d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:49.827053 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56e116-01c2-4232-a98b-0515a64223d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:49.828081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56e116-01c2-4232-a98b-0515a64223d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:49.829707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56e116-01c2-4232-a98b-0515a64223d6 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:49.846187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1076/2160] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:49.847634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f685f-fe18-4444-9c90-a6a1ce15dbad admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:49.861785 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f685f-fe18-4444-9c90-a6a1ce15dbad admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:49.867001 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600b5ec-349d-49b7-b76d-3bcfff4b5197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:49.867754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600b5ec-349d-49b7-b76d-3bcfff4b5197 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:49.868040 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600b5ec-349d-49b7-b76d-3bcfff4b5197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:49.868309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600b5ec-349d-49b7-b76d-3bcfff4b5197 admin admin] PATH_INFO: `/v3/projects/479aeb8f7cf94a97801bb39fba4c74c5/users/0472d84dba934a9c993c944b6436e7c6/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:49.887066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600b5ec-349d-49b7-b76d-3bcfff4b5197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=479aeb8f7cf94a97801bb39fba4c74c5, user_id=0472d84dba934a9c993c944b6436e7c6, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:49.890601 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600b5ec-349d-49b7-b76d-3bcfff4b5197 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:50.062236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1085/2161] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:49 2022] PUT /identity/v3/projects/479aeb8f7cf94a97801bb39fba4c74c5/users/0472d84dba934a9c993c944b6436e7c6/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:50.073355 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab227c9-0c7f-46b3-81fa-97c05b8638d5 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:50.073772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab227c9-0c7f-46b3-81fa-97c05b8638d5 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:50.074052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab227c9-0c7f-46b3-81fa-97c05b8638d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:50.123641 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-7ab227c9-0c7f-46b3-81fa-97c05b8638d5 None None] MFA Rules not processed for user `0472d84dba934a9c993c944b6436e7c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:50.126315 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7ab227c9-0c7f-46b3-81fa-97c05b8638d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:50.254716 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1077/2162] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:50 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:50.259123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600b5ec-349d-49b7-b76d-3bcfff4b5197 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:50.263053 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:50.411216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600b5ec-349d-49b7-b76d-3bcfff4b5197 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:50.430070 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7dee34-c27a-46bb-80db-c58cfb23bb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:50.431126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dee34-c27a-46bb-80db-c58cfb23bb7e admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:50.431635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dee34-c27a-46bb-80db-c58cfb23bb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:50.431984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dee34-c27a-46bb-80db-c58cfb23bb7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:50.433257 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7dee34-c27a-46bb-80db-c58cfb23bb7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:50.439794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7dee34-c27a-46bb-80db-c58cfb23bb7e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:50.476537 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab227c9-0c7f-46b3-81fa-97c05b8638d5 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:50.478157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7ab227c9-0c7f-46b3-81fa-97c05b8638d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:50.501521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab227c9-0c7f-46b3-81fa-97c05b8638d5 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:50.514875 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:50.515564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:50.515846 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:50.516110 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1 admin admin] PATH_INFO: `/v3/projects/12154033004241e4b957c313b74d8667` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:50.522632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12154033004241e4b957c313b74d8667)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:50.546669 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:50.561383 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1086/2163] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:50 2022] POST /identity/v3/projects => generated 399 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:50.567855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7dee34-c27a-46bb-80db-c58cfb23bb7e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:50.706734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7dee34-c27a-46bb-80db-c58cfb23bb7e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:50.712862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad98f4e-b5b2-4b64-b65d-17e07b6eefb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:50.713834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad98f4e-b5b2-4b64-b65d-17e07b6eefb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:50.714147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad98f4e-b5b2-4b64-b65d-17e07b6eefb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:50.714418 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad98f4e-b5b2-4b64-b65d-17e07b6eefb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:50.715247 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad98f4e-b5b2-4b64-b65d-17e07b6eefb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:50.717221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad98f4e-b5b2-4b64-b65d-17e07b6eefb1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:50.744244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12154033004241e4b957c313b74d8667', 'request_id': 'req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:50.752270 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12154033004241e4b957c313b74d8667', 'request_id': 'req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:50.872920 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1087/2164] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:50 2022] POST /identity/v3/users => generated 306 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:50.874326 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad98f4e-b5b2-4b64-b65d-17e07b6eefb1 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:50.892415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad98f4e-b5b2-4b64-b65d-17e07b6eefb1 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:50.895795 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12154033004241e4b957c313b74d8667', 'request_id': 'req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:50.924726 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1 admin admin] The token cache is being invalidate because project 12154033004241e4b957c313b74d8667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:50.925033 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:50.926540 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:50.928665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1078/2165] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:50 2022] DELETE /identity/v3/projects/12154033004241e4b957c313b74d8667 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:26:50.933412 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:50.936020 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:50.997710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606734e-5e9a-430b-860e-7fd33d1a4308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:50.998833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606734e-5e9a-430b-860e-7fd33d1a4308 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:50.999140 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606734e-5e9a-430b-860e-7fd33d1a4308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:50.999447 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606734e-5e9a-430b-860e-7fd33d1a4308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:51.000681 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606734e-5e9a-430b-860e-7fd33d1a4308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:51.002734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606734e-5e9a-430b-860e-7fd33d1a4308 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:51.040375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1088/2166] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:51.043525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e072f-742d-4af6-865b-95bbf2e76260 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:51.043525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e072f-742d-4af6-865b-95bbf2e76260 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:51.043525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e072f-742d-4af6-865b-95bbf2e76260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:51.067067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea29e9-c1f4-4ebc-850b-191b441d09d1 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:51.084253 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ba27f-5624-4044-8620-b1a56d614f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:51.084994 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e37e072f-742d-4af6-865b-95bbf2e76260 None None] MFA Rules not processed for user `28638075e935490bb7c8c229de9d14c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:51.085465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ba27f-5624-4044-8620-b1a56d614f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:51.086160 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ba27f-5624-4044-8620-b1a56d614f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:51.086160 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ba27f-5624-4044-8620-b1a56d614f02 admin admin] PATH_INFO: `/v3/users/59fe590a4b744e1c8270d968773ee77a` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:51.086843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e37e072f-742d-4af6-865b-95bbf2e76260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:51.087542 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ba27f-5624-4044-8620-b1a56d614f02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59fe590a4b744e1c8270d968773ee77a)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:51.090116 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ba27f-5624-4044-8620-b1a56d614f02 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:51.207425 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1089/2167] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:51.224611 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e072f-742d-4af6-865b-95bbf2e76260 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:51.224611 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e37e072f-742d-4af6-865b-95bbf2e76260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:51.230649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e072f-742d-4af6-865b-95bbf2e76260 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:51.236398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e61f3-8d9b-48aa-b026-c6accce4c5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:51.237072 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e61f3-8d9b-48aa-b026-c6accce4c5d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:51.237389 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e61f3-8d9b-48aa-b026-c6accce4c5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:51.237668 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e61f3-8d9b-48aa-b026-c6accce4c5d8 admin admin] PATH_INFO: `/v3/projects/2d7bdc40c4e64a3c9c475e818d0c8236/users/95fa2e11b9f6446fb440a82ce21d5da4/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:51.253776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e61f3-8d9b-48aa-b026-c6accce4c5d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d7bdc40c4e64a3c9c475e818d0c8236, user_id=95fa2e11b9f6446fb440a82ce21d5da4, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:51.256491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e61f3-8d9b-48aa-b026-c6accce4c5d8 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:51.535348 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1090/2168] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:51 2022] PUT /identity/v3/projects/2d7bdc40c4e64a3c9c475e818d0c8236/users/95fa2e11b9f6446fb440a82ce21d5da4/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:51.535348 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e61f3-8d9b-48aa-b026-c6accce4c5d8 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:51.535348 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4a3e61f3-8d9b-48aa-b026-c6accce4c5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:51.607267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e61f3-8d9b-48aa-b026-c6accce4c5d8 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:51.607439 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c70c15-f846-4b1a-b46a-ee8f22578aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:51.608030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c70c15-f846-4b1a-b46a-ee8f22578aac admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:51.608304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c70c15-f846-4b1a-b46a-ee8f22578aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:51.608575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c70c15-f846-4b1a-b46a-ee8f22578aac admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:51.609230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c70c15-f846-4b1a-b46a-ee8f22578aac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:51.611245 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c70c15-f846-4b1a-b46a-ee8f22578aac admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:51.657656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f50ba27f-5624-4044-8620-b1a56d614f02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59fe590a4b744e1c8270d968773ee77a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:51.670466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f50ba27f-5624-4044-8620-b1a56d614f02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59fe590a4b744e1c8270d968773ee77a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:51.681250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f50ba27f-5624-4044-8620-b1a56d614f02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59fe590a4b744e1c8270d968773ee77a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:51.770105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1091/2169] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:51 2022] POST /identity/v3/projects => generated 383 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:51.770105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c70c15-f846-4b1a-b46a-ee8f22578aac admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:51.788638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f50ba27f-5624-4044-8620-b1a56d614f02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59fe590a4b744e1c8270d968773ee77a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:51.789681 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f50ba27f-5624-4044-8620-b1a56d614f02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59fe590a4b744e1c8270d968773ee77a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:51.801433 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1079/2170] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:50 2022] DELETE /identity/v3/users/59fe590a4b744e1c8270d968773ee77a => generated 0 bytes in 869 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:51.807989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ba27f-5624-4044-8620-b1a56d614f02 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:51.810336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f50ba27f-5624-4044-8620-b1a56d614f02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:51.865172 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c70c15-f846-4b1a-b46a-ee8f22578aac admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:51.908339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ba27f-5624-4044-8620-b1a56d614f02 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:51.913449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1162835-110e-4625-ac43-8c5d72ae456b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:51.914123 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1162835-110e-4625-ac43-8c5d72ae456b admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:51.914470 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1162835-110e-4625-ac43-8c5d72ae456b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:51.914754 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1162835-110e-4625-ac43-8c5d72ae456b admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:51.915750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1162835-110e-4625-ac43-8c5d72ae456b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:51.919751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1162835-110e-4625-ac43-8c5d72ae456b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:51.949356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df7342-6e67-456e-bb67-7be805986357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:51.949818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df7342-6e67-456e-bb67-7be805986357 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:51.950048 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df7342-6e67-456e-bb67-7be805986357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:51.950321 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df7342-6e67-456e-bb67-7be805986357 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:51.951074 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df7342-6e67-456e-bb67-7be805986357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:51.953541 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df7342-6e67-456e-bb67-7be805986357 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:51.967208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1092/2171] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:51.971548 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df7342-6e67-456e-bb67-7be805986357 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:51.974395 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-84df7342-6e67-456e-bb67-7be805986357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:51.988201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df7342-6e67-456e-bb67-7be805986357 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:51.996323 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76b3ca-32b3-4653-8887-8abb9aefe2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:51.997104 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76b3ca-32b3-4653-8887-8abb9aefe2cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:51.997458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76b3ca-32b3-4653-8887-8abb9aefe2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:51.999004 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76b3ca-32b3-4653-8887-8abb9aefe2cf admin admin] PATH_INFO: `/v3/projects/2d7bdc40c4e64a3c9c475e818d0c8236/users/95fa2e11b9f6446fb440a82ce21d5da4/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:52.000678 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76b3ca-32b3-4653-8887-8abb9aefe2cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d7bdc40c4e64a3c9c475e818d0c8236, user_id=95fa2e11b9f6446fb440a82ce21d5da4, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:52.007203 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76b3ca-32b3-4653-8887-8abb9aefe2cf admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:52.033190 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1093/2172] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:51 2022] PUT /identity/v3/projects/2d7bdc40c4e64a3c9c475e818d0c8236/users/95fa2e11b9f6446fb440a82ce21d5da4/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:52.035504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6a3e-6342-4c40-a5dd-6b17e45a083f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:52.035841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6a3e-6342-4c40-a5dd-6b17e45a083f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:52.036123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6a3e-6342-4c40-a5dd-6b17e45a083f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:52.046955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1080/2173] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:51 2022] POST /identity/v3/users => generated 298 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:52.048547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1162835-110e-4625-ac43-8c5d72ae456b admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:52.064949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1162835-110e-4625-ac43-8c5d72ae456b admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:52.079656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-44ef6a3e-6342-4c40-a5dd-6b17e45a083f None None] MFA Rules not processed for user `95fa2e11b9f6446fb440a82ce21d5da4`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:52.081983 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-44ef6a3e-6342-4c40-a5dd-6b17e45a083f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:52.165220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bd7ea-d6c2-4749-9887-c3fc47fb3e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:52.166074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bd7ea-d6c2-4749-9887-c3fc47fb3e83 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:52.168050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bd7ea-d6c2-4749-9887-c3fc47fb3e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:52.170333 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bd7ea-d6c2-4749-9887-c3fc47fb3e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:52.171212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743bd7ea-d6c2-4749-9887-c3fc47fb3e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:52.173897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743bd7ea-d6c2-4749-9887-c3fc47fb3e83 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:52.196717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1081/2174] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:52.201683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bd7ea-d6c2-4749-9887-c3fc47fb3e83 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:52.212335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bd7ea-d6c2-4749-9887-c3fc47fb3e83 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:52.226239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721b3ce-f8ee-47bd-b985-f60737796e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:52.232977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721b3ce-f8ee-47bd-b985-f60737796e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:52.233349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721b3ce-f8ee-47bd-b985-f60737796e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:52.233692 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721b3ce-f8ee-47bd-b985-f60737796e35 admin admin] PATH_INFO: `/v3/projects/e3a87f423aab4aeba8f4d971f878514c/users/254d381deace4007a67ecb9ee7b6144b/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:52.256679 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1094/2175] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:52 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:52.257868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4721b3ce-f8ee-47bd-b985-f60737796e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a87f423aab4aeba8f4d971f878514c, user_id=254d381deace4007a67ecb9ee7b6144b, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:52.268125 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4721b3ce-f8ee-47bd-b985-f60737796e35 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:52.269608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef6a3e-6342-4c40-a5dd-6b17e45a083f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:52.271130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-44ef6a3e-6342-4c40-a5dd-6b17e45a083f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:52.346759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef6a3e-6342-4c40-a5dd-6b17e45a083f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:52.373016 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26649803-0df3-4e63-a83b-e86a41a0a0de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:52.386117 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26649803-0df3-4e63-a83b-e86a41a0a0de service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:52.386581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26649803-0df3-4e63-a83b-e86a41a0a0de service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:52.414269 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26649803-0df3-4e63-a83b-e86a41a0a0de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:52.431520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1082/2176] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:52 2022] PUT /identity/v3/projects/e3a87f423aab4aeba8f4d971f878514c/users/254d381deace4007a67ecb9ee7b6144b/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:52.437586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721b3ce-f8ee-47bd-b985-f60737796e35 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:52.445395 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26649803-0df3-4e63-a83b-e86a41a0a0de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:52.447502 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26649803-0df3-4e63-a83b-e86a41a0a0de service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:52.473607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1095/2177] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:52 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:26:52.499948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26649803-0df3-4e63-a83b-e86a41a0a0de service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:52.506144 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-26649803-0df3-4e63-a83b-e86a41a0a0de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:52.534866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26649803-0df3-4e63-a83b-e86a41a0a0de service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:52.622255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ab587-49e3-43e6-a9f8-408f84438f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:52.622255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ab587-49e3-43e6-a9f8-408f84438f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:52.622255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ab587-49e3-43e6-a9f8-408f84438f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:52.622255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ab587-49e3-43e6-a9f8-408f84438f16 admin admin] PATH_INFO: `/v3/projects/069464e6363946ef9efc3e1145c524a6` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:52.623221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ab587-49e3-43e6-a9f8-408f84438f16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=069464e6363946ef9efc3e1145c524a6)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:52.624745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ab587-49e3-43e6-a9f8-408f84438f16 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:52.658900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721b3ce-f8ee-47bd-b985-f60737796e35 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:52.686377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28f517-eb41-4f7a-9bc2-2ddb72a14f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:52.687316 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28f517-eb41-4f7a-9bc2-2ddb72a14f0a admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:52.687913 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28f517-eb41-4f7a-9bc2-2ddb72a14f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:52.688808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28f517-eb41-4f7a-9bc2-2ddb72a14f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:52.689805 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28f517-eb41-4f7a-9bc2-2ddb72a14f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:52.692432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28f517-eb41-4f7a-9bc2-2ddb72a14f0a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:52.705680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1083/2178] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:52.710297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28f517-eb41-4f7a-9bc2-2ddb72a14f0a admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:52.722079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28f517-eb41-4f7a-9bc2-2ddb72a14f0a admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:52.727386 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf253593-130a-4a61-9762-5ebff739b30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:52.728089 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf253593-130a-4a61-9762-5ebff739b30d admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:52.728365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf253593-130a-4a61-9762-5ebff739b30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:52.728644 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf253593-130a-4a61-9762-5ebff739b30d admin admin] PATH_INFO: `/v3/projects/e3a87f423aab4aeba8f4d971f878514c/users/254d381deace4007a67ecb9ee7b6144b/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:52.731418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf253593-130a-4a61-9762-5ebff739b30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a87f423aab4aeba8f4d971f878514c, user_id=254d381deace4007a67ecb9ee7b6144b, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:52.734440 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf253593-130a-4a61-9762-5ebff739b30d admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:52.747434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1084/2179] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:52 2022] PUT /identity/v3/projects/e3a87f423aab4aeba8f4d971f878514c/users/254d381deace4007a67ecb9ee7b6144b/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:26:52.764612 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:52.792369 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf253593-130a-4a61-9762-5ebff739b30d admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:52.803317 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-103ab587-49e3-43e6-a9f8-408f84438f16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '069464e6363946ef9efc3e1145c524a6', 'request_id': 'req-103ab587-49e3-43e6-a9f8-408f84438f16'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:52.871588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf253593-130a-4a61-9762-5ebff739b30d admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:52.884466 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-103ab587-49e3-43e6-a9f8-408f84438f16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '069464e6363946ef9efc3e1145c524a6', 'request_id': 'req-103ab587-49e3-43e6-a9f8-408f84438f16'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:52.895474 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-103ab587-49e3-43e6-a9f8-408f84438f16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '069464e6363946ef9efc3e1145c524a6', 'request_id': 'req-103ab587-49e3-43e6-a9f8-408f84438f16'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:52.925976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-103ab587-49e3-43e6-a9f8-408f84438f16 admin admin] The token cache is being invalidate because project 069464e6363946ef9efc3e1145c524a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:52.926349 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-103ab587-49e3-43e6-a9f8-408f84438f16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:52.928410 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-103ab587-49e3-43e6-a9f8-408f84438f16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:52.932375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1096/2180] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:52 2022] DELETE /identity/v3/projects/069464e6363946ef9efc3e1145c524a6 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:52.939160 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c510da-a692-4f4c-a037-c2e42af27e3b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:52.939998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c510da-a692-4f4c-a037-c2e42af27e3b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:52.940313 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c510da-a692-4f4c-a037-c2e42af27e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:53.002216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-83c510da-a692-4f4c-a037-c2e42af27e3b None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:53.003800 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-83c510da-a692-4f4c-a037-c2e42af27e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:53.006713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec8a656-4bd9-48ca-a396-8c9acb771c05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:53.008631 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8a656-4bd9-48ca-a396-8c9acb771c05 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:53.009179 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8a656-4bd9-48ca-a396-8c9acb771c05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:53.009709 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8a656-4bd9-48ca-a396-8c9acb771c05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:53.027183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-aec8a656-4bd9-48ca-a396-8c9acb771c05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:53.150556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec8a656-4bd9-48ca-a396-8c9acb771c05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:53.153515 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1097/2181] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:53.154263 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec8a656-4bd9-48ca-a396-8c9acb771c05 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:53.154607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7fd5a-20e0-4d58-90b3-9af448fe21af None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:53.155046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7fd5a-20e0-4d58-90b3-9af448fe21af None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:53.155365 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7fd5a-20e0-4d58-90b3-9af448fe21af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:53.169846 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1085/2182] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:52 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:26:53.198259 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-19f7fd5a-20e0-4d58-90b3-9af448fe21af None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:53.199730 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-19f7fd5a-20e0-4d58-90b3-9af448fe21af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:53.212291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1098/2183] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:53.217358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec8a656-4bd9-48ca-a396-8c9acb771c05 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:53.247243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec8a656-4bd9-48ca-a396-8c9acb771c05 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:53.250770 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277fe7e-7c5b-41e6-8bf9-0bced75699f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:53.252838 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277fe7e-7c5b-41e6-8bf9-0bced75699f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:53.252990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277fe7e-7c5b-41e6-8bf9-0bced75699f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:53.253200 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277fe7e-7c5b-41e6-8bf9-0bced75699f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:53.253941 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2277fe7e-7c5b-41e6-8bf9-0bced75699f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:26:53.259248 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2277fe7e-7c5b-41e6-8bf9-0bced75699f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:53.263949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2277fe7e-7c5b-41e6-8bf9-0bced75699f3 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:53.300768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1086/2184] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:26:53.302106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7fd5a-20e0-4d58-90b3-9af448fe21af None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:53.317882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7fd5a-20e0-4d58-90b3-9af448fe21af None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:53.343399 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980a9fb-2f54-41cd-a6d5-0aeab3fe37d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:53.343399 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980a9fb-2f54-41cd-a6d5-0aeab3fe37d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:53.343399 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980a9fb-2f54-41cd-a6d5-0aeab3fe37d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:53.343399 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980a9fb-2f54-41cd-a6d5-0aeab3fe37d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:53.343787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980a9fb-2f54-41cd-a6d5-0aeab3fe37d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:53.344811 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980a9fb-2f54-41cd-a6d5-0aeab3fe37d3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:53.439464 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:53.560845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1099/2185] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:53 2022] POST /identity/v3/projects => generated 381 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:53.564830 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277fe7e-7c5b-41e6-8bf9-0bced75699f3 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:53.582042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277fe7e-7c5b-41e6-8bf9-0bced75699f3 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:53.675432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007fb40-20e9-406e-b1b6-ef6992e76997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:53.678605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007fb40-20e9-406e-b1b6-ef6992e76997 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:53.679011 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007fb40-20e9-406e-b1b6-ef6992e76997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:53.679395 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007fb40-20e9-406e-b1b6-ef6992e76997 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:53.681895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007fb40-20e9-406e-b1b6-ef6992e76997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:53.688793 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007fb40-20e9-406e-b1b6-ef6992e76997 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:53.824302 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1087/2186] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:53 2022] POST /identity/v3/users => generated 291 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:53.829232 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980a9fb-2f54-41cd-a6d5-0aeab3fe37d3 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:53.845565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980a9fb-2f54-41cd-a6d5-0aeab3fe37d3 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:53.861503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0699e-4ada-483a-a191-0c84bc897d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:53.862153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0699e-4ada-483a-a191-0c84bc897d0d admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:53.862441 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0699e-4ada-483a-a191-0c84bc897d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:53.862717 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0699e-4ada-483a-a191-0c84bc897d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:53.864166 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f0699e-4ada-483a-a191-0c84bc897d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:53.865925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f0699e-4ada-483a-a191-0c84bc897d0d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:53.906223 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1100/2187] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:53.907655 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007fb40-20e9-406e-b1b6-ef6992e76997 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:53.932714 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007fb40-20e9-406e-b1b6-ef6992e76997 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:53.946564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cb3d9-6534-4c7a-b227-2cacdbe2ba50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:53.947335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cb3d9-6534-4c7a-b227-2cacdbe2ba50 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:53.949469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cb3d9-6534-4c7a-b227-2cacdbe2ba50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:53.949469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cb3d9-6534-4c7a-b227-2cacdbe2ba50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:53.949469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cb3d9-6534-4c7a-b227-2cacdbe2ba50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:53.952930 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cb3d9-6534-4c7a-b227-2cacdbe2ba50 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:53.986705 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1088/2188] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:26:53.991887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0699e-4ada-483a-a191-0c84bc897d0d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:54.010716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0699e-4ada-483a-a191-0c84bc897d0d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:54.033127 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5cba65-4d22-4681-9a05-c80b31ee7932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:54.033816 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cba65-4d22-4681-9a05-c80b31ee7932 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:54.034097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cba65-4d22-4681-9a05-c80b31ee7932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:54.034372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cba65-4d22-4681-9a05-c80b31ee7932 admin admin] PATH_INFO: `/v3/projects/4be712729c8546d9a1266f27e38632a3/users/59a21c139e23454f8728b0b0bd23bae3/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:54.065130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5cba65-4d22-4681-9a05-c80b31ee7932 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4be712729c8546d9a1266f27e38632a3, user_id=59a21c139e23454f8728b0b0bd23bae3, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:54.068089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5cba65-4d22-4681-9a05-c80b31ee7932 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:54.248890 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1101/2189] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:53 2022] PUT /identity/v3/projects/4be712729c8546d9a1266f27e38632a3/users/59a21c139e23454f8728b0b0bd23bae3/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:54.261885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6de508-482c-4f12-bf79-4d91bc1bd3c5 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:54.262195 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6de508-482c-4f12-bf79-4d91bc1bd3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:54.262466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6de508-482c-4f12-bf79-4d91bc1bd3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:54.346432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-5b6de508-482c-4f12-bf79-4d91bc1bd3c5 None None] MFA Rules not processed for user `59a21c139e23454f8728b0b0bd23bae3`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:54.346432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5b6de508-482c-4f12-bf79-4d91bc1bd3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:54.534615 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1089/2190] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:54.535749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5cba65-4d22-4681-9a05-c80b31ee7932 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:54.736952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5cba65-4d22-4681-9a05-c80b31ee7932 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:54.757551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb8a5a-5865-40f2-b195-49c051a8de5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:54.758195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb8a5a-5865-40f2-b195-49c051a8de5a admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:54.758481 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb8a5a-5865-40f2-b195-49c051a8de5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:54.761211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb8a5a-5865-40f2-b195-49c051a8de5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:54.761982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb8a5a-5865-40f2-b195-49c051a8de5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:54.794331 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb8a5a-5865-40f2-b195-49c051a8de5a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:54.950022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1102/2191] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:54 2022] POST /identity/v3/projects => generated 381 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:54.954099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6de508-482c-4f12-bf79-4d91bc1bd3c5 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:54.974975 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6de508-482c-4f12-bf79-4d91bc1bd3c5 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:55.135140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae018fd-e585-4383-80da-b8c4105d9078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:55.138392 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae018fd-e585-4383-80da-b8c4105d9078 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:55.138392 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae018fd-e585-4383-80da-b8c4105d9078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:55.138392 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae018fd-e585-4383-80da-b8c4105d9078 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:55.138392 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae018fd-e585-4383-80da-b8c4105d9078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:55.140698 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae018fd-e585-4383-80da-b8c4105d9078 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:55.306465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1090/2192] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:54 2022] POST /identity/v3/users => generated 297 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:55.307755 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb8a5a-5865-40f2-b195-49c051a8de5a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:55.326702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb8a5a-5865-40f2-b195-49c051a8de5a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:55.332422 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7715d-41bc-4118-baee-054aaaed01ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:55.333028 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7715d-41bc-4118-baee-054aaaed01ab admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:55.333301 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7715d-41bc-4118-baee-054aaaed01ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:55.333583 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7715d-41bc-4118-baee-054aaaed01ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:55.334528 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd7715d-41bc-4118-baee-054aaaed01ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:55.340378 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd7715d-41bc-4118-baee-054aaaed01ab admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:55.363117 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1103/2193] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:55.367628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae018fd-e585-4383-80da-b8c4105d9078 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:55.379101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae018fd-e585-4383-80da-b8c4105d9078 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:55.383909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bbd1f-e373-4554-8c9f-3c7cd6447510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:55.384585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bbd1f-e373-4554-8c9f-3c7cd6447510 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:55.384870 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bbd1f-e373-4554-8c9f-3c7cd6447510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:55.385216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bbd1f-e373-4554-8c9f-3c7cd6447510 admin admin] PATH_INFO: `/v3/projects/12f7c66c966c4218943d9663c50638d5/users/b21ea7f26277423f9d7a639ddd339094/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:55.402694 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5bbd1f-e373-4554-8c9f-3c7cd6447510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12f7c66c966c4218943d9663c50638d5, user_id=b21ea7f26277423f9d7a639ddd339094, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:55.405421 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5bbd1f-e373-4554-8c9f-3c7cd6447510 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:55.445125 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:55.518247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1091/2194] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:55 2022] PUT /identity/v3/projects/12f7c66c966c4218943d9663c50638d5/users/b21ea7f26277423f9d7a639ddd339094/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:26:55.522124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7715d-41bc-4118-baee-054aaaed01ab admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:55.538943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7715d-41bc-4118-baee-054aaaed01ab admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:55.710183 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3360b-87a2-4756-a09f-4ec129306504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:55.713926 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3360b-87a2-4756-a09f-4ec129306504 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:55.713926 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3360b-87a2-4756-a09f-4ec129306504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:55.713926 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3360b-87a2-4756-a09f-4ec129306504 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:55.713926 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3360b-87a2-4756-a09f-4ec129306504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:55.716014 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3360b-87a2-4756-a09f-4ec129306504 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:55.732496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1104/2195] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:55.733958 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bbd1f-e373-4554-8c9f-3c7cd6447510 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:55.746523 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bbd1f-e373-4554-8c9f-3c7cd6447510 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:55.752288 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f80d58-aec8-4a9d-befd-10ffe4c92bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:55.753233 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f80d58-aec8-4a9d-befd-10ffe4c92bbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:55.753781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f80d58-aec8-4a9d-befd-10ffe4c92bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:55.754215 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f80d58-aec8-4a9d-befd-10ffe4c92bbc admin admin] PATH_INFO: `/v3/projects/12f7c66c966c4218943d9663c50638d5/users/b21ea7f26277423f9d7a639ddd339094/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:55.756668 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f80d58-aec8-4a9d-befd-10ffe4c92bbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=12f7c66c966c4218943d9663c50638d5, user_id=b21ea7f26277423f9d7a639ddd339094, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:55.771495 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f80d58-aec8-4a9d-befd-10ffe4c92bbc admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:55.792737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1092/2196] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:55 2022] PUT /identity/v3/projects/12f7c66c966c4218943d9663c50638d5/users/b21ea7f26277423f9d7a639ddd339094/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:55.793753 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:55.795625 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01b47b-e41a-4d60-9d14-65ca36411c33 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:55.796329 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01b47b-e41a-4d60-9d14-65ca36411c33 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:55.796612 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01b47b-e41a-4d60-9d14-65ca36411c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:55.846268 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ac01b47b-e41a-4d60-9d14-65ca36411c33 None None] MFA Rules not processed for user `b21ea7f26277423f9d7a639ddd339094`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:55.847927 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ac01b47b-e41a-4d60-9d14-65ca36411c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:55.986175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1105/2197] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:56.009510 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f80d58-aec8-4a9d-befd-10ffe4c92bbc admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:56.011511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-79f80d58-aec8-4a9d-befd-10ffe4c92bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:56.123015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f80d58-aec8-4a9d-befd-10ffe4c92bbc admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:56.129295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5de665-9da6-4edb-b90e-bd7b6cfc51e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:56.130329 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5de665-9da6-4edb-b90e-bd7b6cfc51e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:56.130678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5de665-9da6-4edb-b90e-bd7b6cfc51e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:56.131031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5de665-9da6-4edb-b90e-bd7b6cfc51e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:56.154323 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5de665-9da6-4edb-b90e-bd7b6cfc51e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:56.155820 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5de665-9da6-4edb-b90e-bd7b6cfc51e4 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:56.185816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1093/2198] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:56.600447 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01b47b-e41a-4d60-9d14-65ca36411c33 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:56.616406 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01b47b-e41a-4d60-9d14-65ca36411c33 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:56.626038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67d03b-ded9-41a7-a180-27c0836693a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:56.626921 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67d03b-ded9-41a7-a180-27c0836693a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:56.627426 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67d03b-ded9-41a7-a180-27c0836693a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:56.627962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67d03b-ded9-41a7-a180-27c0836693a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:56.757378 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67d03b-ded9-41a7-a180-27c0836693a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:56.763981 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67d03b-ded9-41a7-a180-27c0836693a9 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:56.804858 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1106/2199] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:57.696068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:57.767955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5de665-9da6-4edb-b90e-bd7b6cfc51e4 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:57.871813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5de665-9da6-4edb-b90e-bd7b6cfc51e4 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:57.878570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df582c45-7cab-4231-86c2-e09d34f08a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:57.880027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df582c45-7cab-4231-86c2-e09d34f08a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:57.886164 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df582c45-7cab-4231-86c2-e09d34f08a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:57.886164 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df582c45-7cab-4231-86c2-e09d34f08a53 admin admin] PATH_INFO: `/v3/users/59a21c139e23454f8728b0b0bd23bae3` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:57.886164 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df582c45-7cab-4231-86c2-e09d34f08a53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59a21c139e23454f8728b0b0bd23bae3)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:57.886164 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df582c45-7cab-4231-86c2-e09d34f08a53 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:57.983393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:58.169410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-df582c45-7cab-4231-86c2-e09d34f08a53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59a21c139e23454f8728b0b0bd23bae3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:58.222326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-df582c45-7cab-4231-86c2-e09d34f08a53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59a21c139e23454f8728b0b0bd23bae3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:58.234511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-df582c45-7cab-4231-86c2-e09d34f08a53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59a21c139e23454f8728b0b0bd23bae3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:58.331617 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-df582c45-7cab-4231-86c2-e09d34f08a53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59a21c139e23454f8728b0b0bd23bae3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:58.335366 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-df582c45-7cab-4231-86c2-e09d34f08a53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59a21c139e23454f8728b0b0bd23bae3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:58.342085 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1094/2200] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:57 2022] DELETE /identity/v3/users/59a21c139e23454f8728b0b0bd23bae3 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:58.346605 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d55dfd-9f90-4812-b531-60d23a8bb32e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:58.347204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d55dfd-9f90-4812-b531-60d23a8bb32e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:58.347712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d55dfd-9f90-4812-b531-60d23a8bb32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:58.377235 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-13d55dfd-9f90-4812-b531-60d23a8bb32e None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:58.378611 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-13d55dfd-9f90-4812-b531-60d23a8bb32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:58.465812 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1107/2201] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:58.477027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df582c45-7cab-4231-86c2-e09d34f08a53 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:58.481057 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-df582c45-7cab-4231-86c2-e09d34f08a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:58.574559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df582c45-7cab-4231-86c2-e09d34f08a53 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:58.581090 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628b340-d92a-4e2f-a23d-470ec1bb32a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:58.581843 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628b340-d92a-4e2f-a23d-470ec1bb32a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:58.582222 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628b340-d92a-4e2f-a23d-470ec1bb32a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:58.584745 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628b340-d92a-4e2f-a23d-470ec1bb32a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:58.599698 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628b340-d92a-4e2f-a23d-470ec1bb32a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:58.601502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628b340-d92a-4e2f-a23d-470ec1bb32a8 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:58.617632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1095/2202] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:59.602892 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d55dfd-9f90-4812-b531-60d23a8bb32e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:59.605148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-13d55dfd-9f90-4812-b531-60d23a8bb32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:59.627880 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d55dfd-9f90-4812-b531-60d23a8bb32e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:59.648229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628b340-d92a-4e2f-a23d-470ec1bb32a8 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:59.649123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b391293-dabb-4404-8999-7948b1a7384d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:59.649963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b391293-dabb-4404-8999-7948b1a7384d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:59.650085 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-8628b340-d92a-4e2f-a23d-470ec1bb32a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:59.650422 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b391293-dabb-4404-8999-7948b1a7384d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:59.652940 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b391293-dabb-4404-8999-7948b1a7384d admin admin] PATH_INFO: `/v3/projects/4be712729c8546d9a1266f27e38632a3` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:59.654514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b391293-dabb-4404-8999-7948b1a7384d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4be712729c8546d9a1266f27e38632a3)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:59.658001 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b391293-dabb-4404-8999-7948b1a7384d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:59.673188 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628b340-d92a-4e2f-a23d-470ec1bb32a8 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:59.686617 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa79f8f-9d29-4748-8dbd-eebe53316310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:59.687554 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa79f8f-9d29-4748-8dbd-eebe53316310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:59.688151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa79f8f-9d29-4748-8dbd-eebe53316310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:59.688586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa79f8f-9d29-4748-8dbd-eebe53316310 admin admin] PATH_INFO: `/v3/users/0472d84dba934a9c993c944b6436e7c6` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:59.692702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa79f8f-9d29-4748-8dbd-eebe53316310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0472d84dba934a9c993c944b6436e7c6)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:59.696094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:59.701209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa79f8f-9d29-4748-8dbd-eebe53316310 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:00.061080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1fa79f8f-9d29-4748-8dbd-eebe53316310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0472d84dba934a9c993c944b6436e7c6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:00.078558 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5b391293-dabb-4404-8999-7948b1a7384d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4be712729c8546d9a1266f27e38632a3', 'request_id': 'req-5b391293-dabb-4404-8999-7948b1a7384d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:00.123038 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1fa79f8f-9d29-4748-8dbd-eebe53316310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0472d84dba934a9c993c944b6436e7c6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:00.148300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1fa79f8f-9d29-4748-8dbd-eebe53316310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0472d84dba934a9c993c944b6436e7c6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:00.249912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5b391293-dabb-4404-8999-7948b1a7384d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4be712729c8546d9a1266f27e38632a3', 'request_id': 'req-5b391293-dabb-4404-8999-7948b1a7384d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:00.264246 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5b391293-dabb-4404-8999-7948b1a7384d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4be712729c8546d9a1266f27e38632a3', 'request_id': 'req-5b391293-dabb-4404-8999-7948b1a7384d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:00.282292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5b391293-dabb-4404-8999-7948b1a7384d admin admin] The token cache is being invalidate because project 4be712729c8546d9a1266f27e38632a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:27:00.282894 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5b391293-dabb-4404-8999-7948b1a7384d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:00.283787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5b391293-dabb-4404-8999-7948b1a7384d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:00.285868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1108/2203] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:59 2022] DELETE /identity/v3/projects/4be712729c8546d9a1266f27e38632a3 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:00.289995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b391293-dabb-4404-8999-7948b1a7384d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:00.294858 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5b391293-dabb-4404-8999-7948b1a7384d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:00.381955 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1fa79f8f-9d29-4748-8dbd-eebe53316310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0472d84dba934a9c993c944b6436e7c6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:00.382465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b391293-dabb-4404-8999-7948b1a7384d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:00.384044 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1fa79f8f-9d29-4748-8dbd-eebe53316310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0472d84dba934a9c993c944b6436e7c6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:00.389358 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31c832-e4af-4fff-94ae-bc64ebdbf213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:00.390115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31c832-e4af-4fff-94ae-bc64ebdbf213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:00.390399 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31c832-e4af-4fff-94ae-bc64ebdbf213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:00.390681 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31c832-e4af-4fff-94ae-bc64ebdbf213 admin admin] PATH_INFO: `/v3/users/b21ea7f26277423f9d7a639ddd339094` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:00.392112 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31c832-e4af-4fff-94ae-bc64ebdbf213 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b21ea7f26277423f9d7a639ddd339094)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:00.394588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31c832-e4af-4fff-94ae-bc64ebdbf213 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:00.395458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1096/2204] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:59 2022] DELETE /identity/v3/users/0472d84dba934a9c993c944b6436e7c6 => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:00.405823 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e06959-2f75-4cdf-9d9e-b79d3802c66b None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:00.407424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e06959-2f75-4cdf-9d9e-b79d3802c66b None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:00.410945 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e06959-2f75-4cdf-9d9e-b79d3802c66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:00.479973 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-94e06959-2f75-4cdf-9d9e-b79d3802c66b None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:00.488016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-94e06959-2f75-4cdf-9d9e-b79d3802c66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:00.623422 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:00.628958 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1097/2205] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:27:00.683778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cb31c832-e4af-4fff-94ae-bc64ebdbf213 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b21ea7f26277423f9d7a639ddd339094'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:00.689457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e06959-2f75-4cdf-9d9e-b79d3802c66b None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:00.693280 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-94e06959-2f75-4cdf-9d9e-b79d3802c66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:00.767113 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cb31c832-e4af-4fff-94ae-bc64ebdbf213 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b21ea7f26277423f9d7a639ddd339094'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:00.802924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cb31c832-e4af-4fff-94ae-bc64ebdbf213 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b21ea7f26277423f9d7a639ddd339094'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:00.882471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e06959-2f75-4cdf-9d9e-b79d3802c66b None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:00.886669 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9857ad-1e97-47ab-aa39-745d13cfeec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:00.887440 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9857ad-1e97-47ab-aa39-745d13cfeec1 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:00.887783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9857ad-1e97-47ab-aa39-745d13cfeec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:00.888119 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9857ad-1e97-47ab-aa39-745d13cfeec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:00.908298 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9857ad-1e97-47ab-aa39-745d13cfeec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:00.910434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9857ad-1e97-47ab-aa39-745d13cfeec1 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:00.928064 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1098/2206] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:27:00.939026 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cb31c832-e4af-4fff-94ae-bc64ebdbf213 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b21ea7f26277423f9d7a639ddd339094'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:00.939921 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cb31c832-e4af-4fff-94ae-bc64ebdbf213 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b21ea7f26277423f9d7a639ddd339094'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:00.946792 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1109/2207] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:27:00 2022] DELETE /identity/v3/users/b21ea7f26277423f9d7a639ddd339094 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:01.187429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9857ad-1e97-47ab-aa39-745d13cfeec1 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:01.189112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ea9857ad-1e97-47ab-aa39-745d13cfeec1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:01.214529 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9857ad-1e97-47ab-aa39-745d13cfeec1 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:01.353108 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3e72a-6346-4485-8f41-97e7c41f6627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:01.355510 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3e72a-6346-4485-8f41-97e7c41f6627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:01.355978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3e72a-6346-4485-8f41-97e7c41f6627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:01.357006 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3e72a-6346-4485-8f41-97e7c41f6627 admin admin] PATH_INFO: `/v3/users/72408b32d16f401292ca1506623989d5` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:01.358537 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3e72a-6346-4485-8f41-97e7c41f6627 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72408b32d16f401292ca1506623989d5)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:01.360687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3e72a-6346-4485-8f41-97e7c41f6627 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:01.598398 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-bba3e72a-6346-4485-8f41-97e7c41f6627 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72408b32d16f401292ca1506623989d5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:01.660209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-bba3e72a-6346-4485-8f41-97e7c41f6627 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72408b32d16f401292ca1506623989d5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:01.681041 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-bba3e72a-6346-4485-8f41-97e7c41f6627 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72408b32d16f401292ca1506623989d5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:01.821416 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-bba3e72a-6346-4485-8f41-97e7c41f6627 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72408b32d16f401292ca1506623989d5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:01.823335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-bba3e72a-6346-4485-8f41-97e7c41f6627 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72408b32d16f401292ca1506623989d5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:01.834345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1099/2208] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:27:01 2022] DELETE /identity/v3/users/72408b32d16f401292ca1506623989d5 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:01.841841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2263cc-fdfc-4dd6-800d-7a2853db6dd9 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:01.844177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2263cc-fdfc-4dd6-800d-7a2853db6dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:01.844732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2263cc-fdfc-4dd6-800d-7a2853db6dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:01.919725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-6f2263cc-fdfc-4dd6-800d-7a2853db6dd9 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:01.924358 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6f2263cc-fdfc-4dd6-800d-7a2853db6dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:02.036933 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3e72a-6346-4485-8f41-97e7c41f6627 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:02.038634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-bba3e72a-6346-4485-8f41-97e7c41f6627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:02.059440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1110/2209] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:02.078403 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3e72a-6346-4485-8f41-97e7c41f6627 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:02.089707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c12bc-bbef-4c3b-a442-306a098520f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:02.097250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c12bc-bbef-4c3b-a442-306a098520f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:02.098522 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c12bc-bbef-4c3b-a442-306a098520f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:02.099755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c12bc-bbef-4c3b-a442-306a098520f9 admin admin] PATH_INFO: `/v3/projects/12f7c66c966c4218943d9663c50638d5` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:02.102032 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2263cc-fdfc-4dd6-800d-7a2853db6dd9 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:02.103514 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c12bc-bbef-4c3b-a442-306a098520f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12f7c66c966c4218943d9663c50638d5)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:02.104580 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6f2263cc-fdfc-4dd6-800d-7a2853db6dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:02.106812 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c12bc-bbef-4c3b-a442-306a098520f9 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:02.138967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:02.254917 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2263cc-fdfc-4dd6-800d-7a2853db6dd9 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:02.260266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c223b62-abf8-465c-af53-9c27923a4f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:02.261150 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c223b62-abf8-465c-af53-9c27923a4f90 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:02.261653 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c223b62-abf8-465c-af53-9c27923a4f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:02.262291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c223b62-abf8-465c-af53-9c27923a4f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:02.274316 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c223b62-abf8-465c-af53-9c27923a4f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:02.276661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c223b62-abf8-465c-af53-9c27923a4f90 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:02.296992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1111/2210] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:27:02.300052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c223b62-abf8-465c-af53-9c27923a4f90 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:02.302228 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-917c12bc-bbef-4c3b-a442-306a098520f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12f7c66c966c4218943d9663c50638d5', 'request_id': 'req-917c12bc-bbef-4c3b-a442-306a098520f9'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:02.306322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8c223b62-abf8-465c-af53-9c27923a4f90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:02.314232 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-917c12bc-bbef-4c3b-a442-306a098520f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12f7c66c966c4218943d9663c50638d5', 'request_id': 'req-917c12bc-bbef-4c3b-a442-306a098520f9'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:02.337191 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c223b62-abf8-465c-af53-9c27923a4f90 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:02.419364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:02.420015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:02.420464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:02.420742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe admin admin] PATH_INFO: `/v3/projects/479aeb8f7cf94a97801bb39fba4c74c5` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:02.423116 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=479aeb8f7cf94a97801bb39fba4c74c5)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:02.427266 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-917c12bc-bbef-4c3b-a442-306a098520f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12f7c66c966c4218943d9663c50638d5', 'request_id': 'req-917c12bc-bbef-4c3b-a442-306a098520f9'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:02.444220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-917c12bc-bbef-4c3b-a442-306a098520f9 admin admin] The token cache is being invalidate because project 12f7c66c966c4218943d9663c50638d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:27:02.444915 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-917c12bc-bbef-4c3b-a442-306a098520f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:02.446071 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-917c12bc-bbef-4c3b-a442-306a098520f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:02.451391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1100/2211] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:27:02 2022] DELETE /identity/v3/projects/12f7c66c966c4218943d9663c50638d5 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:02.451889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:02.453449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af5f4f-bbee-4881-91a7-1f44d256b9b6 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:02.456698 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af5f4f-bbee-4881-91a7-1f44d256b9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:02.458773 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af5f4f-bbee-4881-91a7-1f44d256b9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:02.499286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-f8af5f4f-bbee-4881-91a7-1f44d256b9b6 None None] MFA Rules not processed for user `5a8fb193bdd1448eab774bd1eef437c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:02.501095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f8af5f4f-bbee-4881-91a7-1f44d256b9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:02.614625 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '479aeb8f7cf94a97801bb39fba4c74c5', 'request_id': 'req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:02.653298 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1101/2212] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:02.655334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bb421-b05e-4013-8ee8-61fbf5fe1a41 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:02.655334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bb421-b05e-4013-8ee8-61fbf5fe1a41 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:02.655960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bb421-b05e-4013-8ee8-61fbf5fe1a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:02.716315 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-633bb421-b05e-4013-8ee8-61fbf5fe1a41 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:02.718007 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-633bb421-b05e-4013-8ee8-61fbf5fe1a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:02.752892 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '479aeb8f7cf94a97801bb39fba4c74c5', 'request_id': 'req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:02.762178 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '479aeb8f7cf94a97801bb39fba4c74c5', 'request_id': 'req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:02.846519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe admin admin] The token cache is being invalidate because project 479aeb8f7cf94a97801bb39fba4c74c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:27:02.846519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:02.847791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:02.852616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1112/2213] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:27:02 2022] DELETE /identity/v3/projects/479aeb8f7cf94a97801bb39fba4c74c5 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:02.857025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:02.861399 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:02.863542 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1102/2214] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:02.864415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bb421-b05e-4013-8ee8-61fbf5fe1a41 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:02.870130 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-633bb421-b05e-4013-8ee8-61fbf5fe1a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:02.897394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bb421-b05e-4013-8ee8-61fbf5fe1a41 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:02.911469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc59f7-41ba-4f2e-b48f-ef6c5fb1eca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:02.912385 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc59f7-41ba-4f2e-b48f-ef6c5fb1eca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:02.913000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc59f7-41ba-4f2e-b48f-ef6c5fb1eca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:02.913464 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc59f7-41ba-4f2e-b48f-ef6c5fb1eca0 admin admin] PATH_INFO: `/v3/users/95fa2e11b9f6446fb440a82ce21d5da4` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:02.914879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc59f7-41ba-4f2e-b48f-ef6c5fb1eca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95fa2e11b9f6446fb440a82ce21d5da4)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:02.917160 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc59f7-41ba-4f2e-b48f-ef6c5fb1eca0 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:03.074916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba04c6-acc6-4bc5-aff6-b9695d68d3fe admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:03.089344 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4c192-3836-4d50-a0f4-f56a5a1c9ae2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:03.090333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4c192-3836-4d50-a0f4-f56a5a1c9ae2 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:03.090852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4c192-3836-4d50-a0f4-f56a5a1c9ae2 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:03.091530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4c192-3836-4d50-a0f4-f56a5a1c9ae2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:03.093948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e3a4c192-3836-4d50-a0f4-f56a5a1c9ae2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:03.145226 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-34dc59f7-41ba-4f2e-b48f-ef6c5fb1eca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95fa2e11b9f6446fb440a82ce21d5da4'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:03.178231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-34dc59f7-41ba-4f2e-b48f-ef6c5fb1eca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95fa2e11b9f6446fb440a82ce21d5da4'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:03.187126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4c192-3836-4d50-a0f4-f56a5a1c9ae2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:03.188879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4c192-3836-4d50-a0f4-f56a5a1c9ae2 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:03.199907 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-34dc59f7-41ba-4f2e-b48f-ef6c5fb1eca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95fa2e11b9f6446fb440a82ce21d5da4'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:03.213509 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1113/2215] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:27:03.217872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baceb6b-7fe2-44dc-aa3b-5cb489774e65 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:03.217872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baceb6b-7fe2-44dc-aa3b-5cb489774e65 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:03.217872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baceb6b-7fe2-44dc-aa3b-5cb489774e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:03.316910 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-34dc59f7-41ba-4f2e-b48f-ef6c5fb1eca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95fa2e11b9f6446fb440a82ce21d5da4'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:03.316910 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-34dc59f7-41ba-4f2e-b48f-ef6c5fb1eca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95fa2e11b9f6446fb440a82ce21d5da4'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:03.321756 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1103/2216] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:27:02 2022] DELETE /identity/v3/users/95fa2e11b9f6446fb440a82ce21d5da4 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:03.474312 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8baceb6b-7fe2-44dc-aa3b-5cb489774e65 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:03.475709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8baceb6b-7fe2-44dc-aa3b-5cb489774e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:03.630405 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1114/2217] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:03.634734 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc59f7-41ba-4f2e-b48f-ef6c5fb1eca0 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:03.649716 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc59f7-41ba-4f2e-b48f-ef6c5fb1eca0 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:03.660183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f48092-302b-4230-8e43-3f54e99caf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:03.660824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f48092-302b-4230-8e43-3f54e99caf56 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:03.661102 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f48092-302b-4230-8e43-3f54e99caf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:03.661556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f48092-302b-4230-8e43-3f54e99caf56 admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:03.662607 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f48092-302b-4230-8e43-3f54e99caf56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:27:03.663069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f48092-302b-4230-8e43-3f54e99caf56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:03.664891 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f48092-302b-4230-8e43-3f54e99caf56 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:03.698566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1104/2218] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:27:03.706909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baceb6b-7fe2-44dc-aa3b-5cb489774e65 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:03.719555 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baceb6b-7fe2-44dc-aa3b-5cb489774e65 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:03.726527 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e27bb-6d98-46a2-9505-d7e238cbd398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:03.727772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e27bb-6d98-46a2-9505-d7e238cbd398 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:03.728144 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e27bb-6d98-46a2-9505-d7e238cbd398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:03.730476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e27bb-6d98-46a2-9505-d7e238cbd398 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:03.730476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e27bb-6d98-46a2-9505-d7e238cbd398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:03.733430 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e27bb-6d98-46a2-9505-d7e238cbd398 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:03.848177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1115/2219] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:27:03 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:27:03.849524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f48092-302b-4230-8e43-3f54e99caf56 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:03.866010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f48092-302b-4230-8e43-3f54e99caf56 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:03.975589 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7883d-498b-41cf-b801-de96d062b19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:03.976633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7883d-498b-41cf-b801-de96d062b19f admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:03.977129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7883d-498b-41cf-b801-de96d062b19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:03.977665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7883d-498b-41cf-b801-de96d062b19f admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:03.978581 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7883d-498b-41cf-b801-de96d062b19f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:03.981438 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7883d-498b-41cf-b801-de96d062b19f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:04.056418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:04.066778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e27bb-6d98-46a2-9505-d7e238cbd398 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:04.070548 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-217e27bb-6d98-46a2-9505-d7e238cbd398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:04.090872 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1105/2220] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:27:03 2022] POST /identity/v3/users => generated 288 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:27:04.091360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e27bb-6d98-46a2-9505-d7e238cbd398 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:04.094556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7883d-498b-41cf-b801-de96d062b19f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:04.121845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f80cd-65a7-4423-8d3d-b0ab5958b200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:04.122584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f80cd-65a7-4423-8d3d-b0ab5958b200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:04.122868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f80cd-65a7-4423-8d3d-b0ab5958b200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:04.123139 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f80cd-65a7-4423-8d3d-b0ab5958b200 admin admin] PATH_INFO: `/v3/projects/3c1d53a091884e578f12887f1357a3bd` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:04.130316 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7883d-498b-41cf-b801-de96d062b19f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:04.138304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f80cd-65a7-4423-8d3d-b0ab5958b200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c1d53a091884e578f12887f1357a3bd)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:04.141984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f49134-0a38-4e54-87e4-bc0108ded69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:04.142334 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f80cd-65a7-4423-8d3d-b0ab5958b200 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:04.143723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f49134-0a38-4e54-87e4-bc0108ded69b admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:04.144205 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f49134-0a38-4e54-87e4-bc0108ded69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:04.144826 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f49134-0a38-4e54-87e4-bc0108ded69b admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:04.147090 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f49134-0a38-4e54-87e4-bc0108ded69b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:04.152952 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f49134-0a38-4e54-87e4-bc0108ded69b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:04.166929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1106/2221] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:27:04.173554 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f49134-0a38-4e54-87e4-bc0108ded69b admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:04.209829 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f49134-0a38-4e54-87e4-bc0108ded69b admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:04.224672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22035ae-f707-49c9-82ec-92d4421742d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:04.227721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22035ae-f707-49c9-82ec-92d4421742d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:04.230200 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22035ae-f707-49c9-82ec-92d4421742d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:04.231076 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22035ae-f707-49c9-82ec-92d4421742d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:04.232095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22035ae-f707-49c9-82ec-92d4421742d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:04.235361 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22035ae-f707-49c9-82ec-92d4421742d1 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:04.252536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1107/2222] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:27:04.253723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22035ae-f707-49c9-82ec-92d4421742d1 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:04.258070 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e22035ae-f707-49c9-82ec-92d4421742d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:04.289441 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22035ae-f707-49c9-82ec-92d4421742d1 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:04.299007 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e59ce8-7432-4b8a-abc0-4756489b24c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:04.299923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e59ce8-7432-4b8a-abc0-4756489b24c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:04.300898 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e59ce8-7432-4b8a-abc0-4756489b24c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:04.301271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e59ce8-7432-4b8a-abc0-4756489b24c8 admin admin] PATH_INFO: `/v3/projects/2d7bdc40c4e64a3c9c475e818d0c8236` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:04.302598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e59ce8-7432-4b8a-abc0-4756489b24c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d7bdc40c4e64a3c9c475e818d0c8236)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:04.305319 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e59ce8-7432-4b8a-abc0-4756489b24c8 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:04.316160 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:04.372156 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e19f80cd-65a7-4423-8d3d-b0ab5958b200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c1d53a091884e578f12887f1357a3bd', 'request_id': 'req-e19f80cd-65a7-4423-8d3d-b0ab5958b200'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:04.496305 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e19f80cd-65a7-4423-8d3d-b0ab5958b200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c1d53a091884e578f12887f1357a3bd', 'request_id': 'req-e19f80cd-65a7-4423-8d3d-b0ab5958b200'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:04.509809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e19f80cd-65a7-4423-8d3d-b0ab5958b200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c1d53a091884e578f12887f1357a3bd', 'request_id': 'req-e19f80cd-65a7-4423-8d3d-b0ab5958b200'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:04.536585 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e19f80cd-65a7-4423-8d3d-b0ab5958b200 admin admin] The token cache is being invalidate because project 3c1d53a091884e578f12887f1357a3bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:27:04.538065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e19f80cd-65a7-4423-8d3d-b0ab5958b200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:04.539322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e19f80cd-65a7-4423-8d3d-b0ab5958b200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:04.547379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1116/2223] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:27:04 2022] DELETE /identity/v3/projects/3c1d53a091884e578f12887f1357a3bd => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:27:04.547379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f80cd-65a7-4423-8d3d-b0ab5958b200 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:04.549211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e19f80cd-65a7-4423-8d3d-b0ab5958b200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:04.556338 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-49e59ce8-7432-4b8a-abc0-4756489b24c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d7bdc40c4e64a3c9c475e818d0c8236', 'request_id': 'req-49e59ce8-7432-4b8a-abc0-4756489b24c8'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:04.583146 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-49e59ce8-7432-4b8a-abc0-4756489b24c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d7bdc40c4e64a3c9c475e818d0c8236', 'request_id': 'req-49e59ce8-7432-4b8a-abc0-4756489b24c8'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:04.656339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-49e59ce8-7432-4b8a-abc0-4756489b24c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d7bdc40c4e64a3c9c475e818d0c8236', 'request_id': 'req-49e59ce8-7432-4b8a-abc0-4756489b24c8'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:04.676873 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-49e59ce8-7432-4b8a-abc0-4756489b24c8 admin admin] The token cache is being invalidate because project 2d7bdc40c4e64a3c9c475e818d0c8236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:27:04.677169 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-49e59ce8-7432-4b8a-abc0-4756489b24c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:04.679505 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-49e59ce8-7432-4b8a-abc0-4756489b24c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:04.687147 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1108/2224] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:27:04 2022] DELETE /identity/v3/projects/2d7bdc40c4e64a3c9c475e818d0c8236 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:04.688901 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a9a67-6703-4313-8790-368a021d484a None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:04.689338 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a9a67-6703-4313-8790-368a021d484a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:04.689749 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a9a67-6703-4313-8790-368a021d484a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:04.693787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f80cd-65a7-4423-8d3d-b0ab5958b200 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:04.733058 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-971a9a67-6703-4313-8790-368a021d484a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:04.735541 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-971a9a67-6703-4313-8790-368a021d484a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:04.836131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8c2c6-df2f-4f37-b7d2-bd69348818c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:04.837233 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8c2c6-df2f-4f37-b7d2-bd69348818c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:04.837521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8c2c6-df2f-4f37-b7d2-bd69348818c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:04.839775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8c2c6-df2f-4f37-b7d2-bd69348818c9 admin admin] PATH_INFO: `/v3/projects/c1c18a3cd9c34c2ea6c1426218303c45/users/02e19e62ea784fd0a483f799193cc72c/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:04.903408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa8c2c6-df2f-4f37-b7d2-bd69348818c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c18a3cd9c34c2ea6c1426218303c45, user_id=02e19e62ea784fd0a483f799193cc72c, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:04.910873 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa8c2c6-df2f-4f37-b7d2-bd69348818c9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:04.926212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1109/2225] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:04.929246 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e99cef-bc57-4a42-9def-6e82ba75cb75 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:04.930932 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e99cef-bc57-4a42-9def-6e82ba75cb75 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:04.931406 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e99cef-bc57-4a42-9def-6e82ba75cb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:04.961810 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-60e99cef-bc57-4a42-9def-6e82ba75cb75 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:04.963967 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-60e99cef-bc57-4a42-9def-6e82ba75cb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:04.982910 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1110/2226] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:04.982910 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade4c52-8e6b-4f81-aa23-a91c36222b61 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:04.982910 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade4c52-8e6b-4f81-aa23-a91c36222b61 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:04.982910 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade4c52-8e6b-4f81-aa23-a91c36222b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:05.038290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-2ade4c52-8e6b-4f81-aa23-a91c36222b61 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:05.039673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2ade4c52-8e6b-4f81-aa23-a91c36222b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:05.054762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1117/2227] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:27:04 2022] PUT /identity/v3/projects/c1c18a3cd9c34c2ea6c1426218303c45/users/02e19e62ea784fd0a483f799193cc72c/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:05.057431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186224bc-578e-4acf-9a6e-4b3412df8e3e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:05.059398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186224bc-578e-4acf-9a6e-4b3412df8e3e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:05.059398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186224bc-578e-4acf-9a6e-4b3412df8e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:05.116081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-186224bc-578e-4acf-9a6e-4b3412df8e3e None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:05.118015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-186224bc-578e-4acf-9a6e-4b3412df8e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:05.177715 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1111/2228] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:05.180907 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e6c39-7465-40ff-b278-fddd996a5e21 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:05.183192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e6c39-7465-40ff-b278-fddd996a5e21 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:05.183959 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e6c39-7465-40ff-b278-fddd996a5e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:05.266689 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-5b0e6c39-7465-40ff-b278-fddd996a5e21 None None] MFA Rules not processed for user `02e19e62ea784fd0a483f799193cc72c`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:05.273010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5b0e6c39-7465-40ff-b278-fddd996a5e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:05.291779 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1118/2229] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:05.297973 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186224bc-578e-4acf-9a6e-4b3412df8e3e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:05.327661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186224bc-578e-4acf-9a6e-4b3412df8e3e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:05.335064 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f831a-e2b7-457e-9601-8e7c9204a08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:05.336139 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f831a-e2b7-457e-9601-8e7c9204a08b admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:05.336620 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f831a-e2b7-457e-9601-8e7c9204a08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:05.337260 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f831a-e2b7-457e-9601-8e7c9204a08b admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:05.340035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f831a-e2b7-457e-9601-8e7c9204a08b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:27:05.340767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f831a-e2b7-457e-9601-8e7c9204a08b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:05.342874 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f831a-e2b7-457e-9601-8e7c9204a08b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:05.356718 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1112/2230] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:05.357873 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e6c39-7465-40ff-b278-fddd996a5e21 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:05.372294 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e6c39-7465-40ff-b278-fddd996a5e21 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:05.379858 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a25d6d-a089-481a-bd12-76a2edf65290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:05.380670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a25d6d-a089-481a-bd12-76a2edf65290 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:05.381070 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a25d6d-a089-481a-bd12-76a2edf65290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:05.381454 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a25d6d-a089-481a-bd12-76a2edf65290 admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:05.382302 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a25d6d-a089-481a-bd12-76a2edf65290 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:27:05.382722 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a25d6d-a089-481a-bd12-76a2edf65290 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:05.383787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1119/2231] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:27:05.385346 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a25d6d-a089-481a-bd12-76a2edf65290 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:05.386455 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f831a-e2b7-457e-9601-8e7c9204a08b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:05.392143 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-486f831a-e2b7-457e-9601-8e7c9204a08b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:05.402106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1113/2232] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:27:05.402106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a25d6d-a089-481a-bd12-76a2edf65290 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:05.414078 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a25d6d-a089-481a-bd12-76a2edf65290 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:05.420477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968b103-9ac3-4265-853d-e2c727b0250f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:05.420477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968b103-9ac3-4265-853d-e2c727b0250f admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:05.421062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968b103-9ac3-4265-853d-e2c727b0250f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:05.421382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968b103-9ac3-4265-853d-e2c727b0250f admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:05.422357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968b103-9ac3-4265-853d-e2c727b0250f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:05.424454 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968b103-9ac3-4265-853d-e2c727b0250f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:05.545716 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1114/2233] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:27:05 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:27:05.546672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968b103-9ac3-4265-853d-e2c727b0250f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:05.603328 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f831a-e2b7-457e-9601-8e7c9204a08b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:05.717643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968b103-9ac3-4265-853d-e2c727b0250f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:05.739076 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc030206-7dbb-4f6d-90e3-e0278fcfb671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:05.739818 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc030206-7dbb-4f6d-90e3-e0278fcfb671 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:05.740244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc030206-7dbb-4f6d-90e3-e0278fcfb671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:05.741156 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc030206-7dbb-4f6d-90e3-e0278fcfb671 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:05.745413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc030206-7dbb-4f6d-90e3-e0278fcfb671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:05.746658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b648e0f-d8f3-4bf2-bd33-ae045efbe630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:05.747595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b648e0f-d8f3-4bf2-bd33-ae045efbe630 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:05.747866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b648e0f-d8f3-4bf2-bd33-ae045efbe630 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:05.748136 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b648e0f-d8f3-4bf2-bd33-ae045efbe630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:05.749654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc030206-7dbb-4f6d-90e3-e0278fcfb671 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:05.876317 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b648e0f-d8f3-4bf2-bd33-ae045efbe630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:05.883300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b648e0f-d8f3-4bf2-bd33-ae045efbe630 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:05.883475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1115/2234] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:27:05 2022] POST /identity/v3/projects => generated 383 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:27:05.883475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc030206-7dbb-4f6d-90e3-e0278fcfb671 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:05.899404 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1120/2235] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:27:05.900612 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b648e0f-d8f3-4bf2-bd33-ae045efbe630 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:05.926918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b648e0f-d8f3-4bf2-bd33-ae045efbe630 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:06.000406 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc030206-7dbb-4f6d-90e3-e0278fcfb671 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:06.008242 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b04144-700c-45ba-ab9c-673535dea3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:06.009239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b04144-700c-45ba-ab9c-673535dea3d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:06.009729 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b04144-700c-45ba-ab9c-673535dea3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:06.010202 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b04144-700c-45ba-ab9c-673535dea3d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:06.011036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b04144-700c-45ba-ab9c-673535dea3d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:06.014261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b04144-700c-45ba-ab9c-673535dea3d5 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:06.107816 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d73382-1ef5-4336-af4b-9fa5e3622d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:06.108446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d73382-1ef5-4336-af4b-9fa5e3622d40 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:06.111734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d73382-1ef5-4336-af4b-9fa5e3622d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:06.112068 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d73382-1ef5-4336-af4b-9fa5e3622d40 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:06.112764 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d73382-1ef5-4336-af4b-9fa5e3622d40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:06.123912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d73382-1ef5-4336-af4b-9fa5e3622d40 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:06.151154 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1116/2236] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:27:05 2022] POST /identity/v3/users => generated 286 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:27:06.154681 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b04144-700c-45ba-ab9c-673535dea3d5 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:06.209031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b04144-700c-45ba-ab9c-673535dea3d5 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:06.216396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44efd742-7daf-4d4b-bfc3-5ebc51994eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:06.217756 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efd742-7daf-4d4b-bfc3-5ebc51994eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:06.218107 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efd742-7daf-4d4b-bfc3-5ebc51994eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:06.219982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efd742-7daf-4d4b-bfc3-5ebc51994eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:06.220771 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44efd742-7daf-4d4b-bfc3-5ebc51994eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:06.222415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44efd742-7daf-4d4b-bfc3-5ebc51994eb6 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:06.259752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1117/2237] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:27:06.264361 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44efd742-7daf-4d4b-bfc3-5ebc51994eb6 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:06.284281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44efd742-7daf-4d4b-bfc3-5ebc51994eb6 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:06.303036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52ffbe-088a-4b0f-8457-73bc77e8d372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:06.303036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52ffbe-088a-4b0f-8457-73bc77e8d372 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:06.303036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52ffbe-088a-4b0f-8457-73bc77e8d372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:06.303036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52ffbe-088a-4b0f-8457-73bc77e8d372 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:06.303706 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52ffbe-088a-4b0f-8457-73bc77e8d372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:06.306026 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52ffbe-088a-4b0f-8457-73bc77e8d372 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:06.321492 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1118/2238] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:27:06.325559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52ffbe-088a-4b0f-8457-73bc77e8d372 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:06.367828 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1121/2239] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:27:05 2022] POST /identity/v3/users => generated 292 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:27:06.369317 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec7d81-5690-4702-8fcb-3aa226679746 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:06.369606 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec7d81-5690-4702-8fcb-3aa226679746 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:06.369882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec7d81-5690-4702-8fcb-3aa226679746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:06.372199 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52ffbe-088a-4b0f-8457-73bc77e8d372 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:06.396022 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d3070-2b16-4291-b501-0da9b002b45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:06.397052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d3070-2b16-4291-b501-0da9b002b45c admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:06.397507 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d3070-2b16-4291-b501-0da9b002b45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:06.397788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d3070-2b16-4291-b501-0da9b002b45c admin admin] PATH_INFO: `/v3/projects/4f82c67c47c243f594cd2bab7ac861aa/users/792178b0829f49c2b5729aeee1805e77/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:06.436994 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-69ec7d81-5690-4702-8fcb-3aa226679746 None None] MFA Rules not processed for user `254d381deace4007a67ecb9ee7b6144b`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:06.438248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-69ec7d81-5690-4702-8fcb-3aa226679746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:06.438369 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d3070-2b16-4291-b501-0da9b002b45c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f82c67c47c243f594cd2bab7ac861aa, user_id=792178b0829f49c2b5729aeee1805e77, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:06.441318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d3070-2b16-4291-b501-0da9b002b45c admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:06.449939 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:06.568211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1122/2240] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:06.569805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec7d81-5690-4702-8fcb-3aa226679746 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:06.575408 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1119/2241] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:27:06 2022] PUT /identity/v3/projects/4f82c67c47c243f594cd2bab7ac861aa/users/792178b0829f49c2b5729aeee1805e77/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:06.576607 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d3070-2b16-4291-b501-0da9b002b45c admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:06.583062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-780d3070-2b16-4291-b501-0da9b002b45c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:06.586358 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec7d81-5690-4702-8fcb-3aa226679746 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:06.707323 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5d850-381a-4cb9-aeb9-3ae54af2a42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:06.708284 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5d850-381a-4cb9-aeb9-3ae54af2a42d admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:06.708578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5d850-381a-4cb9-aeb9-3ae54af2a42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:06.708855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5d850-381a-4cb9-aeb9-3ae54af2a42d admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:06.709521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5d850-381a-4cb9-aeb9-3ae54af2a42d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:06.711204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5d850-381a-4cb9-aeb9-3ae54af2a42d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:06.731086 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d3070-2b16-4291-b501-0da9b002b45c admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:06.733265 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed342ea7-e0c3-43f3-ac6a-6692f416dc12 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558e3b6dd9754c57ad028aa431a68b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:06.734132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed342ea7-e0c3-43f3-ac6a-6692f416dc12 service glance] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:06.734434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed342ea7-e0c3-43f3-ac6a-6692f416dc12 service glance] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:06.734777 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed342ea7-e0c3-43f3-ac6a-6692f416dc12 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:06.737473 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ed342ea7-e0c3-43f3-ac6a-6692f416dc12 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:06.749814 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1123/2242] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:27:06.751563 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f63b8-d6dd-4ff8-a856-1d4d227a780a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:06.751997 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f63b8-d6dd-4ff8-a856-1d4d227a780a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:06.752301 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f63b8-d6dd-4ff8-a856-1d4d227a780a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:06.776468 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-273f63b8-d6dd-4ff8-a856-1d4d227a780a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:06.777787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-273f63b8-d6dd-4ff8-a856-1d4d227a780a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:06.786320 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1124/2243] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:06.787195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f63b8-d6dd-4ff8-a856-1d4d227a780a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:06.805239 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f63b8-d6dd-4ff8-a856-1d4d227a780a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:06.834103 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed342ea7-e0c3-43f3-ac6a-6692f416dc12 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:06.834103 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed342ea7-e0c3-43f3-ac6a-6692f416dc12 service glance] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:06.841733 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dcae3-5e04-4581-bac7-2764a2bd0e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:06.844907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dcae3-5e04-4581-bac7-2764a2bd0e66 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:06.845379 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dcae3-5e04-4581-bac7-2764a2bd0e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:06.845779 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1120/2244] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:27:06.846365 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dcae3-5e04-4581-bac7-2764a2bd0e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:06.846644 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed342ea7-e0c3-43f3-ac6a-6692f416dc12 service glance] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:06.848138 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dcae3-5e04-4581-bac7-2764a2bd0e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:06.855139 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dcae3-5e04-4581-bac7-2764a2bd0e66 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:06.864237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ed342ea7-e0c3-43f3-ac6a-6692f416dc12 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:06.873220 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1125/2245] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:27:06.877206 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dcae3-5e04-4581-bac7-2764a2bd0e66 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:06.892109 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dcae3-5e04-4581-bac7-2764a2bd0e66 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:06.898009 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fefc92-449b-481f-bdb7-f7a8134a9255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:06.899009 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fefc92-449b-481f-bdb7-f7a8134a9255 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:06.908899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fefc92-449b-481f-bdb7-f7a8134a9255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:06.908899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fefc92-449b-481f-bdb7-f7a8134a9255 admin admin] PATH_INFO: `/v3/projects/562bec9e36a048b0a6d6952ffa5409ee/users/c9f9e702a46a4828a7286b4e6da7ffff/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:06.931482 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fefc92-449b-481f-bdb7-f7a8134a9255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=562bec9e36a048b0a6d6952ffa5409ee, user_id=c9f9e702a46a4828a7286b4e6da7ffff, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:06.941529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fefc92-449b-481f-bdb7-f7a8134a9255 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:07.014321 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed342ea7-e0c3-43f3-ac6a-6692f416dc12 service glance] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:07.043437 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1126/2246] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:27:06 2022] PUT /identity/v3/projects/562bec9e36a048b0a6d6952ffa5409ee/users/c9f9e702a46a4828a7286b4e6da7ffff/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:07.044978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824addba-8c8f-4327-9f6d-d7ada647e62a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:07.045265 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824addba-8c8f-4327-9f6d-d7ada647e62a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:07.045548 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824addba-8c8f-4327-9f6d-d7ada647e62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:07.113071 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-824addba-8c8f-4327-9f6d-d7ada647e62a None None] MFA Rules not processed for user `c9f9e702a46a4828a7286b4e6da7ffff`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:07.114820 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-824addba-8c8f-4327-9f6d-d7ada647e62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:07.190544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb526c-e044-44f9-9618-b5d2a817feb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:07.191472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb526c-e044-44f9-9618-b5d2a817feb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:07.191472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb526c-e044-44f9-9618-b5d2a817feb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:07.191950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb526c-e044-44f9-9618-b5d2a817feb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:07.305948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb526c-e044-44f9-9618-b5d2a817feb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:07.309677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb526c-e044-44f9-9618-b5d2a817feb6 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:07.317871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1127/2247] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:07.324145 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824addba-8c8f-4327-9f6d-d7ada647e62a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:07.361091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1121/2248] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:27:07.370368 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824addba-8c8f-4327-9f6d-d7ada647e62a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:07.384062 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb955b8-164c-4647-92a7-1fdb9fb43ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:07.385829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb955b8-164c-4647-92a7-1fdb9fb43ee8 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:07.388677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb955b8-164c-4647-92a7-1fdb9fb43ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:07.389095 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb955b8-164c-4647-92a7-1fdb9fb43ee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:07.389879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb955b8-164c-4647-92a7-1fdb9fb43ee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:07.395244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc95a32-38c4-444c-8bbc-cde7030e4e3e None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:07.395656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc95a32-38c4-444c-8bbc-cde7030e4e3e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:07.396039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc95a32-38c4-444c-8bbc-cde7030e4e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:07.405536 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb955b8-164c-4647-92a7-1fdb9fb43ee8 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:07.490270 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-1bc95a32-38c4-444c-8bbc-cde7030e4e3e None None] MFA Rules not processed for user `5a667fa07b1148998959835d9995ad60`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:07.493371 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1bc95a32-38c4-444c-8bbc-cde7030e4e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:07.501963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:07.587111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1128/2249] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:27:07 2022] POST /identity/v3/projects => generated 383 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:27:07.589531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb955b8-164c-4647-92a7-1fdb9fb43ee8 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:07.659357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1122/2250] 10.222.0.78 () {58 vars in 1009 bytes} [Mon Sep 19 11:27:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 19 11:27:07.721002 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc95a32-38c4-444c-8bbc-cde7030e4e3e None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:07.724307 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1bc95a32-38c4-444c-8bbc-cde7030e4e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:07.740336 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb955b8-164c-4647-92a7-1fdb9fb43ee8 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:07.754970 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c244468-d9b8-4328-bd07-c74f5045dce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:07.758328 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c244468-d9b8-4328-bd07-c74f5045dce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:07.758631 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c244468-d9b8-4328-bd07-c74f5045dce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:07.759305 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c244468-d9b8-4328-bd07-c74f5045dce9 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:07.762461 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c244468-d9b8-4328-bd07-c74f5045dce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:07.764554 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c244468-d9b8-4328-bd07-c74f5045dce9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:07.898089 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc95a32-38c4-444c-8bbc-cde7030e4e3e None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:07.937455 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1129/2251] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:27:07 2022] POST /identity/v3/users => generated 298 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:27:07.941404 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c244468-d9b8-4328-bd07-c74f5045dce9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:07.962084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:07.969797 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c244468-d9b8-4328-bd07-c74f5045dce9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:07.984088 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b1666-8a20-43b8-b7d0-8e8d51edeed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:07.986235 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b1666-8a20-43b8-b7d0-8e8d51edeed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:07.989945 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b1666-8a20-43b8-b7d0-8e8d51edeed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:07.990451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b1666-8a20-43b8-b7d0-8e8d51edeed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:07.991377 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b1666-8a20-43b8-b7d0-8e8d51edeed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:07.995806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b1666-8a20-43b8-b7d0-8e8d51edeed0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:08.026001 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22b0f3-b6d8-49bc-9114-e1f43b321d93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf462b2aa3fc4e2ca58851549cf0c644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:08.030754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1130/2252] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:27:08.035437 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b1666-8a20-43b8-b7d0-8e8d51edeed0 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:08.039992 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22b0f3-b6d8-49bc-9114-e1f43b321d93 service swift] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:08.040504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22b0f3-b6d8-49bc-9114-e1f43b321d93 service swift] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:08.041687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22b0f3-b6d8-49bc-9114-e1f43b321d93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:08.048981 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b1666-8a20-43b8-b7d0-8e8d51edeed0 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:08.059079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee6378-28f7-4926-ba70-349e4398dfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:08.060047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee6378-28f7-4926-ba70-349e4398dfb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:08.060483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee6378-28f7-4926-ba70-349e4398dfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:08.060909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee6378-28f7-4926-ba70-349e4398dfb6 admin admin] PATH_INFO: `/v3/projects/7ca18ee7c26c433fb8aa2c9496449c16/users/54693b33c49b4a70860f208946027ef9/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:08.084210 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:08.093886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee6378-28f7-4926-ba70-349e4398dfb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca18ee7c26c433fb8aa2c9496449c16, user_id=54693b33c49b4a70860f208946027ef9, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:08.100495 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee6378-28f7-4926-ba70-349e4398dfb6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:08.210899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1131/2253] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:27:08 2022] PUT /identity/v3/projects/7ca18ee7c26c433fb8aa2c9496449c16/users/54693b33c49b4a70860f208946027ef9/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:08.216000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee6378-28f7-4926-ba70-349e4398dfb6 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:08.224929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22b0f3-b6d8-49bc-9114-e1f43b321d93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:08.225901 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22b0f3-b6d8-49bc-9114-e1f43b321d93 service swift] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:08.270747 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1123/2254] 10.222.0.78 () {60 vars in 1384 bytes} [Mon Sep 19 11:27:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:27:08.380925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee6378-28f7-4926-ba70-349e4398dfb6 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:08.390682 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d687cb-e6a1-475e-a8fb-3760dfc3a6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:08.394259 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d687cb-e6a1-475e-a8fb-3760dfc3a6ff admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:08.394676 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d687cb-e6a1-475e-a8fb-3760dfc3a6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:08.395057 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d687cb-e6a1-475e-a8fb-3760dfc3a6ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:08.396258 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d687cb-e6a1-475e-a8fb-3760dfc3a6ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:08.398375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d687cb-e6a1-475e-a8fb-3760dfc3a6ff admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:08.424151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22b0f3-b6d8-49bc-9114-e1f43b321d93 service swift] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:08.425154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1132/2255] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:27:08.442922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22b0f3-b6d8-49bc-9114-e1f43b321d93 service swift] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:08.458875 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e0f1f-ea7b-4281-9513-dfcb91d19490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:08.464219 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e0f1f-ea7b-4281-9513-dfcb91d19490 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:08.464525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e0f1f-ea7b-4281-9513-dfcb91d19490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:08.464780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e0f1f-ea7b-4281-9513-dfcb91d19490 admin admin] PATH_INFO: `/v3/projects/7ca18ee7c26c433fb8aa2c9496449c16/users/54693b33c49b4a70860f208946027ef9/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:08.474489 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e0f1f-ea7b-4281-9513-dfcb91d19490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca18ee7c26c433fb8aa2c9496449c16, user_id=54693b33c49b4a70860f208946027ef9, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:08.486289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e0f1f-ea7b-4281-9513-dfcb91d19490 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:08.534677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1124/2256] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:27:08 2022] PUT /identity/v3/projects/7ca18ee7c26c433fb8aa2c9496449c16/users/54693b33c49b4a70860f208946027ef9/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:27:08.537534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a87595-c6b0-403d-bab9-b42e2f7f0134 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:08.537823 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a87595-c6b0-403d-bab9-b42e2f7f0134 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:08.538098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a87595-c6b0-403d-bab9-b42e2f7f0134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:08.596231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-40a87595-c6b0-403d-bab9-b42e2f7f0134 None None] MFA Rules not processed for user `54693b33c49b4a70860f208946027ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:08.598953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-40a87595-c6b0-403d-bab9-b42e2f7f0134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:08.846761 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1133/2257] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:08 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:08.939817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e0f1f-ea7b-4281-9513-dfcb91d19490 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:09.073371 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e0f1f-ea7b-4281-9513-dfcb91d19490 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:09.086276 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd29fc-a1d7-4f30-992a-5383205c73d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:09.086645 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd29fc-a1d7-4f30-992a-5383205c73d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:09.086929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd29fc-a1d7-4f30-992a-5383205c73d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:09.087223 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd29fc-a1d7-4f30-992a-5383205c73d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:09.236133 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd29fc-a1d7-4f30-992a-5383205c73d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:09.237611 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd29fc-a1d7-4f30-992a-5383205c73d6 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:09.262500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1125/2258] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:27:09.297106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cc118-bf7f-40c9-912f-9cdc34337e2d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:09.298105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cc118-bf7f-40c9-912f-9cdc34337e2d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:09.298569 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cc118-bf7f-40c9-912f-9cdc34337e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:09.353822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-566cc118-bf7f-40c9-912f-9cdc34337e2d None None] MFA Rules not processed for user `5a8fb193bdd1448eab774bd1eef437c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:09.355471 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-566cc118-bf7f-40c9-912f-9cdc34337e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:09.473531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1134/2259] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:09.484653 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:09.712003 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:11.188046 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e84a3c-df86-4ced-9202-926107eba2d8 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:11.188563 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e84a3c-df86-4ced-9202-926107eba2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:11.188671 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e84a3c-df86-4ced-9202-926107eba2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:11.231633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-43e84a3c-df86-4ced-9202-926107eba2d8 None None] MFA Rules not processed for user `d6fee905418f4d879da460b6758d4f23`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:11.234152 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-43e84a3c-df86-4ced-9202-926107eba2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:11.411816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1126/2260] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:13.347730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:13.499906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566cc118-bf7f-40c9-912f-9cdc34337e2d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:13.504038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-566cc118-bf7f-40c9-912f-9cdc34337e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:13.564767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:13.838079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566cc118-bf7f-40c9-912f-9cdc34337e2d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:13.871093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718c905-156c-429f-aa14-284306543aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:13.872206 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718c905-156c-429f-aa14-284306543aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:13.872784 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718c905-156c-429f-aa14-284306543aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:13.873263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718c905-156c-429f-aa14-284306543aec admin admin] PATH_INFO: `/v3/users/02e19e62ea784fd0a483f799193cc72c` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:13.879469 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718c905-156c-429f-aa14-284306543aec admin admin] RBAC: Authorizing `identity:delete_user(user_id=02e19e62ea784fd0a483f799193cc72c)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:13.882611 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718c905-156c-429f-aa14-284306543aec admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:14.265281 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9718c905-156c-429f-aa14-284306543aec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02e19e62ea784fd0a483f799193cc72c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:14.324186 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9718c905-156c-429f-aa14-284306543aec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02e19e62ea784fd0a483f799193cc72c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:14.348790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9718c905-156c-429f-aa14-284306543aec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02e19e62ea784fd0a483f799193cc72c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:14.455040 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9718c905-156c-429f-aa14-284306543aec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02e19e62ea784fd0a483f799193cc72c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:14.455838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9718c905-156c-429f-aa14-284306543aec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02e19e62ea784fd0a483f799193cc72c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:14.465094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1135/2261] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:27:13 2022] DELETE /identity/v3/users/02e19e62ea784fd0a483f799193cc72c => generated 0 bytes in 966 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:14.469699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55981b23-101c-44dd-837d-b929e651b16e None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:14.470604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55981b23-101c-44dd-837d-b929e651b16e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:14.471038 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55981b23-101c-44dd-837d-b929e651b16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:14.513043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-55981b23-101c-44dd-837d-b929e651b16e None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:14.514870 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-55981b23-101c-44dd-837d-b929e651b16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:14.721043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1127/2262] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:27:14.726358 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718c905-156c-429f-aa14-284306543aec admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:14.735672 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9718c905-156c-429f-aa14-284306543aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:14.886771 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718c905-156c-429f-aa14-284306543aec admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:14.907258 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd097cd1-28de-4ff8-a8b9-c87d124a2066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:14.908140 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd097cd1-28de-4ff8-a8b9-c87d124a2066 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:14.911486 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd097cd1-28de-4ff8-a8b9-c87d124a2066 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:14.911863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd097cd1-28de-4ff8-a8b9-c87d124a2066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:14.948587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd097cd1-28de-4ff8-a8b9-c87d124a2066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:14.952391 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd097cd1-28de-4ff8-a8b9-c87d124a2066 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:14.971582 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1136/2263] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:27:16.205960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55981b23-101c-44dd-837d-b929e651b16e None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:16.207748 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-55981b23-101c-44dd-837d-b929e651b16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:16.236255 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55981b23-101c-44dd-837d-b929e651b16e None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:16.267742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a5e43-f639-4be1-8d3e-c3e21000c97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:16.347593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a5e43-f639-4be1-8d3e-c3e21000c97d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:16.351162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a5e43-f639-4be1-8d3e-c3e21000c97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:16.357374 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a5e43-f639-4be1-8d3e-c3e21000c97d admin admin] PATH_INFO: `/v3/projects/c1c18a3cd9c34c2ea6c1426218303c45` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:16.365914 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a5e43-f639-4be1-8d3e-c3e21000c97d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1c18a3cd9c34c2ea6c1426218303c45)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:16.380131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a5e43-f639-4be1-8d3e-c3e21000c97d admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:16.614696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-934a5e43-f639-4be1-8d3e-c3e21000c97d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1c18a3cd9c34c2ea6c1426218303c45', 'request_id': 'req-934a5e43-f639-4be1-8d3e-c3e21000c97d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:16.630874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-934a5e43-f639-4be1-8d3e-c3e21000c97d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1c18a3cd9c34c2ea6c1426218303c45', 'request_id': 'req-934a5e43-f639-4be1-8d3e-c3e21000c97d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:16.690769 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:16.752696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-934a5e43-f639-4be1-8d3e-c3e21000c97d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1c18a3cd9c34c2ea6c1426218303c45', 'request_id': 'req-934a5e43-f639-4be1-8d3e-c3e21000c97d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:16.781356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:16.801107 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-934a5e43-f639-4be1-8d3e-c3e21000c97d admin admin] The token cache is being invalidate because project c1c18a3cd9c34c2ea6c1426218303c45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:27:16.801416 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-934a5e43-f639-4be1-8d3e-c3e21000c97d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:16.804825 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-934a5e43-f639-4be1-8d3e-c3e21000c97d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:16.812660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1128/2264] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:27:16 2022] DELETE /identity/v3/projects/c1c18a3cd9c34c2ea6c1426218303c45 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:16.817290 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3a3f9-7c65-4a9b-9e6d-332b6aeddfb9 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:16.817646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3a3f9-7c65-4a9b-9e6d-332b6aeddfb9 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:16.820566 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3a3f9-7c65-4a9b-9e6d-332b6aeddfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:16.847580 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f3d3a3f9-7c65-4a9b-9e6d-332b6aeddfb9 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:16.849236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f3d3a3f9-7c65-4a9b-9e6d-332b6aeddfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:16.993721 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1137/2265] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:16.995517 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800f766-9bb9-421c-bfbc-a31e06284e5c None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:16.995953 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800f766-9bb9-421c-bfbc-a31e06284e5c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:17.000872 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800f766-9bb9-421c-bfbc-a31e06284e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:17.063660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-2800f766-9bb9-421c-bfbc-a31e06284e5c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:17.064795 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2800f766-9bb9-421c-bfbc-a31e06284e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:17.103461 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3a3f9-7c65-4a9b-9e6d-332b6aeddfb9 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:17.111248 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1129/2266] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:17.140502 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3a3f9-7c65-4a9b-9e6d-332b6aeddfb9 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:17.161199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5623119-a659-4eb2-a915-a7a20a9da33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:17.161843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5623119-a659-4eb2-a915-a7a20a9da33d admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:17.162133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5623119-a659-4eb2-a915-a7a20a9da33d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:17.162400 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5623119-a659-4eb2-a915-a7a20a9da33d admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:17.163295 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5623119-a659-4eb2-a915-a7a20a9da33d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:27:17.163613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5623119-a659-4eb2-a915-a7a20a9da33d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:17.165247 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5623119-a659-4eb2-a915-a7a20a9da33d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:17.209638 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1138/2267] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:27:17.210677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800f766-9bb9-421c-bfbc-a31e06284e5c None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:17.222812 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800f766-9bb9-421c-bfbc-a31e06284e5c None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:17.231616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c8de8-b5a5-4d2f-996a-0d886f880ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:17.232726 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c8de8-b5a5-4d2f-996a-0d886f880ead admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:17.233273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c8de8-b5a5-4d2f-996a-0d886f880ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:17.233905 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c8de8-b5a5-4d2f-996a-0d886f880ead admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:17.234787 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c8de8-b5a5-4d2f-996a-0d886f880ead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:17.237003 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c8de8-b5a5-4d2f-996a-0d886f880ead admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:17.368606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1130/2268] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:27:17 2022] POST /identity/v3/projects => generated 397 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:27:17.372756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5623119-a659-4eb2-a915-a7a20a9da33d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:17.396307 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5623119-a659-4eb2-a915-a7a20a9da33d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:17.602083 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022739ac-7e68-48b9-8d65-8a59e9e6faa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:17.604813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022739ac-7e68-48b9-8d65-8a59e9e6faa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:17.605081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022739ac-7e68-48b9-8d65-8a59e9e6faa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:17.605358 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022739ac-7e68-48b9-8d65-8a59e9e6faa5 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:17.606029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022739ac-7e68-48b9-8d65-8a59e9e6faa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:17.613483 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022739ac-7e68-48b9-8d65-8a59e9e6faa5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:17.758986 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1139/2269] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:27:17 2022] POST /identity/v3/users => generated 299 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:27:17.768577 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c8de8-b5a5-4d2f-996a-0d886f880ead admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:17.803307 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c8de8-b5a5-4d2f-996a-0d886f880ead admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:17.816319 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c169c98-4d69-41d0-9a49-c14af4da4f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:17.819371 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c169c98-4d69-41d0-9a49-c14af4da4f1a admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:17.819905 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c169c98-4d69-41d0-9a49-c14af4da4f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:17.821004 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c169c98-4d69-41d0-9a49-c14af4da4f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:17.821942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c169c98-4d69-41d0-9a49-c14af4da4f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:17.829229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c169c98-4d69-41d0-9a49-c14af4da4f1a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:17.858392 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1131/2270] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:27:17.862169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022739ac-7e68-48b9-8d65-8a59e9e6faa5 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:17.883750 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022739ac-7e68-48b9-8d65-8a59e9e6faa5 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:17.905555 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcf661-04ce-471a-b302-5cf2417b0dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:17.906389 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcf661-04ce-471a-b302-5cf2417b0dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:17.906820 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcf661-04ce-471a-b302-5cf2417b0dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:17.907568 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcf661-04ce-471a-b302-5cf2417b0dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:17.910342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbcf661-04ce-471a-b302-5cf2417b0dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:17.935814 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbcf661-04ce-471a-b302-5cf2417b0dd4 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:17.984419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1140/2271] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:27:17.985885 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c169c98-4d69-41d0-9a49-c14af4da4f1a admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:18.003438 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c169c98-4d69-41d0-9a49-c14af4da4f1a admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:18.020978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81665546-8c1b-4c18-8893-8f87de75273c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:18.021780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81665546-8c1b-4c18-8893-8f87de75273c admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:18.027546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81665546-8c1b-4c18-8893-8f87de75273c admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:18.029428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81665546-8c1b-4c18-8893-8f87de75273c admin admin] PATH_INFO: `/v3/projects/b4dd02db0e39427f9a67d9d69443a8bc/users/750af9f8c7e14816a4c4a3eabdd3274d/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:18.079367 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcf661-04ce-471a-b302-5cf2417b0dd4 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:18.084051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6fbcf661-04ce-471a-b302-5cf2417b0dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:18.094101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81665546-8c1b-4c18-8893-8f87de75273c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4dd02db0e39427f9a67d9d69443a8bc, user_id=750af9f8c7e14816a4c4a3eabdd3274d, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:18.101125 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81665546-8c1b-4c18-8893-8f87de75273c admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:18.195240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1132/2272] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:27:17 2022] PUT /identity/v3/projects/b4dd02db0e39427f9a67d9d69443a8bc/users/750af9f8c7e14816a4c4a3eabdd3274d/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:27:18.206224 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb233418-a1a4-4646-a19e-3e923503898c None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:18.206754 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb233418-a1a4-4646-a19e-3e923503898c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:18.207383 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb233418-a1a4-4646-a19e-3e923503898c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:18.248676 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcf661-04ce-471a-b302-5cf2417b0dd4 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:18.273731 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-cb233418-a1a4-4646-a19e-3e923503898c None None] MFA Rules not processed for user `750af9f8c7e14816a4c4a3eabdd3274d`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:18.280109 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-cb233418-a1a4-4646-a19e-3e923503898c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:18.372656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099ed7d-f79a-4742-8578-35e327e55a82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:18.373619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099ed7d-f79a-4742-8578-35e327e55a82 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:18.374048 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099ed7d-f79a-4742-8578-35e327e55a82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:18.374487 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099ed7d-f79a-4742-8578-35e327e55a82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:18.381118 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d099ed7d-f79a-4742-8578-35e327e55a82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:18.498682 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099ed7d-f79a-4742-8578-35e327e55a82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:18.501742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099ed7d-f79a-4742-8578-35e327e55a82 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:18.519218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1141/2273] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:27:18.661903 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1133/2274] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:18 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:27:18.855945 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099ed7d-f79a-4742-8578-35e327e55a82 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:18.858711 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d099ed7d-f79a-4742-8578-35e327e55a82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:18.874488 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099ed7d-f79a-4742-8578-35e327e55a82 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:18.904729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae5885-1524-437c-a364-90a43704cc87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:18.905490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae5885-1524-437c-a364-90a43704cc87 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:18.908417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae5885-1524-437c-a364-90a43704cc87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:18.910715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae5885-1524-437c-a364-90a43704cc87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:18.946337 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae5885-1524-437c-a364-90a43704cc87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:18.948940 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae5885-1524-437c-a364-90a43704cc87 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:18.960956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1142/2275] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:18 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:27:19.776164 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:19.965706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:21.170615 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:21.616946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:22.333816 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:22.615900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:22.917539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c88207-0882-4c49-a395-97dec285f97a None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:22.918141 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c88207-0882-4c49-a395-97dec285f97a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:22.918141 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c88207-0882-4c49-a395-97dec285f97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:22.968235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-e3c88207-0882-4c49-a395-97dec285f97a None None] MFA Rules not processed for user `792178b0829f49c2b5729aeee1805e77`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:22.968235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e3c88207-0882-4c49-a395-97dec285f97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:23.057746 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1134/2276] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:23.063127 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae5885-1524-437c-a364-90a43704cc87 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:23.064891 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-abae5885-1524-437c-a364-90a43704cc87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:23.170382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae5885-1524-437c-a364-90a43704cc87 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:23.176867 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a920b3-92cc-4bd1-aa1a-33f869cea152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:23.177827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a920b3-92cc-4bd1-aa1a-33f869cea152 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:23.178328 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a920b3-92cc-4bd1-aa1a-33f869cea152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:23.178809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a920b3-92cc-4bd1-aa1a-33f869cea152 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:23.179746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a920b3-92cc-4bd1-aa1a-33f869cea152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:23.183905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a920b3-92cc-4bd1-aa1a-33f869cea152 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:23.288818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1143/2277] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:27:23 2022] POST /identity/v3/projects => generated 371 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:27:23.289920 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c88207-0882-4c49-a395-97dec285f97a None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:23.308984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c88207-0882-4c49-a395-97dec285f97a None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:23.419047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9c724-2db6-447b-a556-b3d40c4f5f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:23.426634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9c724-2db6-447b-a556-b3d40c4f5f8e admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:23.426634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9c724-2db6-447b-a556-b3d40c4f5f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:23.426634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9c724-2db6-447b-a556-b3d40c4f5f8e admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:23.426634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece9c724-2db6-447b-a556-b3d40c4f5f8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:23.426634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece9c724-2db6-447b-a556-b3d40c4f5f8e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:23.518882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1135/2278] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:27:23 2022] POST /identity/v3/users => generated 292 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:27:23.520957 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a920b3-92cc-4bd1-aa1a-33f869cea152 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:23.538564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a920b3-92cc-4bd1-aa1a-33f869cea152 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:23.547709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94f629-94b6-4b3b-808f-c5c9290b579b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:23.547709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94f629-94b6-4b3b-808f-c5c9290b579b admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:23.547709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94f629-94b6-4b3b-808f-c5c9290b579b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:23.547709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94f629-94b6-4b3b-808f-c5c9290b579b admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:23.549913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94f629-94b6-4b3b-808f-c5c9290b579b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:23.553144 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94f629-94b6-4b3b-808f-c5c9290b579b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:23.556118 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:23.589067 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1144/2279] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:27:23.593636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9c724-2db6-447b-a556-b3d40c4f5f8e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:23.626916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9c724-2db6-447b-a556-b3d40c4f5f8e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:23.646610 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042e1c5-bf25-4f75-a556-d808f72b06a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:23.648436 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042e1c5-bf25-4f75-a556-d808f72b06a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:23.648760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042e1c5-bf25-4f75-a556-d808f72b06a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:23.649051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042e1c5-bf25-4f75-a556-d808f72b06a5 admin admin] PATH_INFO: `/v3/projects/cd13f6e839454a7792487439e155534c/users/f9e2834bd72946aa8d6655d2937f7eb4/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:23.702786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042e1c5-bf25-4f75-a556-d808f72b06a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd13f6e839454a7792487439e155534c, user_id=f9e2834bd72946aa8d6655d2937f7eb4, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:23.706571 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042e1c5-bf25-4f75-a556-d808f72b06a5 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:23.789157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1136/2280] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:27:23 2022] PUT /identity/v3/projects/cd13f6e839454a7792487439e155534c/users/f9e2834bd72946aa8d6655d2937f7eb4/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:23.793057 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94f629-94b6-4b3b-808f-c5c9290b579b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:23.809451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94f629-94b6-4b3b-808f-c5c9290b579b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:23.920037 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:23.987850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e74d0e-c7ba-49e9-b506-fe5567f85223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:23.990106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e74d0e-c7ba-49e9-b506-fe5567f85223 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:23.990746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e74d0e-c7ba-49e9-b506-fe5567f85223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:23.992467 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e74d0e-c7ba-49e9-b506-fe5567f85223 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:23.994339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e74d0e-c7ba-49e9-b506-fe5567f85223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:23.998787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e74d0e-c7ba-49e9-b506-fe5567f85223 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:24.031890 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1145/2281] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:27:24.033182 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042e1c5-bf25-4f75-a556-d808f72b06a5 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:24.058416 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042e1c5-bf25-4f75-a556-d808f72b06a5 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:24.077786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3148eaa-3a55-449b-a0f9-853e35dd47b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:24.078554 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3148eaa-3a55-449b-a0f9-853e35dd47b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:24.079114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3148eaa-3a55-449b-a0f9-853e35dd47b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:24.080198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3148eaa-3a55-449b-a0f9-853e35dd47b0 admin admin] PATH_INFO: `/v3/projects/cd13f6e839454a7792487439e155534c/users/f9e2834bd72946aa8d6655d2937f7eb4/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:24.086100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3148eaa-3a55-449b-a0f9-853e35dd47b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd13f6e839454a7792487439e155534c, user_id=f9e2834bd72946aa8d6655d2937f7eb4, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:24.090367 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3148eaa-3a55-449b-a0f9-853e35dd47b0 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:24.163209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1137/2282] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:27:24 2022] PUT /identity/v3/projects/cd13f6e839454a7792487439e155534c/users/f9e2834bd72946aa8d6655d2937f7eb4/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:24.325860 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e864f8f-0e2c-431c-b296-f4258ec9c399 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:24.326157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e864f8f-0e2c-431c-b296-f4258ec9c399 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:24.326457 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e864f8f-0e2c-431c-b296-f4258ec9c399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:24.379075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4e864f8f-0e2c-431c-b296-f4258ec9c399 None None] MFA Rules not processed for user `5a667fa07b1148998959835d9995ad60`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:24.380617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4e864f8f-0e2c-431c-b296-f4258ec9c399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:24.519315 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1146/2283] 10.222.0.78 () {58 vars in 1009 bytes} [Mon Sep 19 11:27:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:24.547081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3148eaa-3a55-449b-a0f9-853e35dd47b0 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:24.548391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f3148eaa-3a55-449b-a0f9-853e35dd47b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:24.641948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3148eaa-3a55-449b-a0f9-853e35dd47b0 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:24.647374 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e890a9-3494-4e4d-a1ba-14228b75385a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf462b2aa3fc4e2ca58851549cf0c644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:24.648099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e890a9-3494-4e4d-a1ba-14228b75385a service swift] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:24.648386 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e890a9-3494-4e4d-a1ba-14228b75385a service swift] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:24.649102 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e890a9-3494-4e4d-a1ba-14228b75385a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:24.657791 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e890a9-3494-4e4d-a1ba-14228b75385a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:24.659532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e890a9-3494-4e4d-a1ba-14228b75385a service swift] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:24.669674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1138/2284] 10.222.0.78 () {60 vars in 1384 bytes} [Mon Sep 19 11:27:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:27:24.707121 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:25.090106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:25.476869 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:26.746964 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:27.261795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:29.026562 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:29.113567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:31.205571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e864f8f-0e2c-431c-b296-f4258ec9c399 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:31.207679 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4e864f8f-0e2c-431c-b296-f4258ec9c399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:31.491913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e864f8f-0e2c-431c-b296-f4258ec9c399 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:31.498192 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef38020-63ae-4be8-8399-e6e605d93023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:31.498940 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef38020-63ae-4be8-8399-e6e605d93023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:31.499358 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef38020-63ae-4be8-8399-e6e605d93023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:31.499723 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef38020-63ae-4be8-8399-e6e605d93023 admin admin] PATH_INFO: `/v3/users/c9f9e702a46a4828a7286b4e6da7ffff` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:31.500973 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef38020-63ae-4be8-8399-e6e605d93023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9f9e702a46a4828a7286b4e6da7ffff)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:31.503029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef38020-63ae-4be8-8399-e6e605d93023 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:31.857440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-fef38020-63ae-4be8-8399-e6e605d93023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9f9e702a46a4828a7286b4e6da7ffff'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:31.877678 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:31.881701 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-fef38020-63ae-4be8-8399-e6e605d93023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9f9e702a46a4828a7286b4e6da7ffff'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:31.891762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-fef38020-63ae-4be8-8399-e6e605d93023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9f9e702a46a4828a7286b4e6da7ffff'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:32.067094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-fef38020-63ae-4be8-8399-e6e605d93023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9f9e702a46a4828a7286b4e6da7ffff'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:32.067691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-fef38020-63ae-4be8-8399-e6e605d93023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9f9e702a46a4828a7286b4e6da7ffff'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:32.076172 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1147/2285] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:27:31 2022] DELETE /identity/v3/users/c9f9e702a46a4828a7286b4e6da7ffff => generated 0 bytes in 871 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:32.081264 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9cdf0-971c-416f-afa3-1f96893770da None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:32.081922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9cdf0-971c-416f-afa3-1f96893770da None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:32.081922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9cdf0-971c-416f-afa3-1f96893770da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:32.102444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-c1f9cdf0-971c-416f-afa3-1f96893770da None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:32.103718 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c1f9cdf0-971c-416f-afa3-1f96893770da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:32.136201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e76a10-b063-4957-99d1-81b7a70dc262 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:32.136635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e76a10-b063-4957-99d1-81b7a70dc262 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:32.137007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e76a10-b063-4957-99d1-81b7a70dc262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:32.170265 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d6e76a10-b063-4957-99d1-81b7a70dc262 None None] MFA Rules not processed for user `28638075e935490bb7c8c229de9d14c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:32.172203 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d6e76a10-b063-4957-99d1-81b7a70dc262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:32.268375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1148/2286] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:27:32.297795 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1139/2287] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:32.312896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e76a10-b063-4957-99d1-81b7a70dc262 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:32.315414 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d6e76a10-b063-4957-99d1-81b7a70dc262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:32.367935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9cdf0-971c-416f-afa3-1f96893770da None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:32.369631 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c1f9cdf0-971c-416f-afa3-1f96893770da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:32.386654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9cdf0-971c-416f-afa3-1f96893770da None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:32.405547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63b0e3-5914-49fc-a416-4e98e0d958b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:32.407379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63b0e3-5914-49fc-a416-4e98e0d958b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:32.409923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63b0e3-5914-49fc-a416-4e98e0d958b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:32.410593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63b0e3-5914-49fc-a416-4e98e0d958b0 admin admin] PATH_INFO: `/v3/users/750af9f8c7e14816a4c4a3eabdd3274d` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:32.418484 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e63b0e3-5914-49fc-a416-4e98e0d958b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=750af9f8c7e14816a4c4a3eabdd3274d)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:32.421982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e63b0e3-5914-49fc-a416-4e98e0d958b0 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:32.464913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e76a10-b063-4957-99d1-81b7a70dc262 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:32.470885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47c328-54b2-4c9c-8704-2e7ac7a67f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:32.471616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47c328-54b2-4c9c-8704-2e7ac7a67f56 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:32.472226 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47c328-54b2-4c9c-8704-2e7ac7a67f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:32.472396 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47c328-54b2-4c9c-8704-2e7ac7a67f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:32.484774 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47c328-54b2-4c9c-8704-2e7ac7a67f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:32.487703 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47c328-54b2-4c9c-8704-2e7ac7a67f56 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:32.498661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1149/2288] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:27:32.702761 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-6e63b0e3-5914-49fc-a416-4e98e0d958b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '750af9f8c7e14816a4c4a3eabdd3274d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:32.755200 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-6e63b0e3-5914-49fc-a416-4e98e0d958b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '750af9f8c7e14816a4c4a3eabdd3274d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:32.784723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-6e63b0e3-5914-49fc-a416-4e98e0d958b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '750af9f8c7e14816a4c4a3eabdd3274d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:32.855347 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-6e63b0e3-5914-49fc-a416-4e98e0d958b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '750af9f8c7e14816a4c4a3eabdd3274d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:32.855994 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-6e63b0e3-5914-49fc-a416-4e98e0d958b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '750af9f8c7e14816a4c4a3eabdd3274d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:32.857784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:32.863778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1140/2289] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:27:32 2022] DELETE /identity/v3/users/750af9f8c7e14816a4c4a3eabdd3274d => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:32.868833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81406489-ea66-4f2a-b457-56f0f7b598ae None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:32.869120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81406489-ea66-4f2a-b457-56f0f7b598ae None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:32.869400 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81406489-ea66-4f2a-b457-56f0f7b598ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:32.906177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-81406489-ea66-4f2a-b457-56f0f7b598ae None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:32.907645 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-81406489-ea66-4f2a-b457-56f0f7b598ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:33.045873 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1150/2290] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:33.071563 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63b0e3-5914-49fc-a416-4e98e0d958b0 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:33.073260 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-6e63b0e3-5914-49fc-a416-4e98e0d958b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:33.182779 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63b0e3-5914-49fc-a416-4e98e0d958b0 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:33.188171 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4f80b-9a7f-4e63-9a1d-387628f8de21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:33.189332 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4f80b-9a7f-4e63-9a1d-387628f8de21 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:33.189805 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4f80b-9a7f-4e63-9a1d-387628f8de21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:33.190304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4f80b-9a7f-4e63-9a1d-387628f8de21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:33.203790 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4f80b-9a7f-4e63-9a1d-387628f8de21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:33.205909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4f80b-9a7f-4e63-9a1d-387628f8de21 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:33.240426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1141/2291] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:27:33.267995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81406489-ea66-4f2a-b457-56f0f7b598ae None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:33.269677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-81406489-ea66-4f2a-b457-56f0f7b598ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:33.297530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81406489-ea66-4f2a-b457-56f0f7b598ae None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:33.311072 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb211bb6-8881-4a49-8135-422295dda8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:33.312847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb211bb6-8881-4a49-8135-422295dda8e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:33.313312 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb211bb6-8881-4a49-8135-422295dda8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:33.314432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb211bb6-8881-4a49-8135-422295dda8e8 admin admin] PATH_INFO: `/v3/projects/562bec9e36a048b0a6d6952ffa5409ee` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:33.318908 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb211bb6-8881-4a49-8135-422295dda8e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=562bec9e36a048b0a6d6952ffa5409ee)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:33.324181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb211bb6-8881-4a49-8135-422295dda8e8 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:33.365795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:33.502413 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-eb211bb6-8881-4a49-8135-422295dda8e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '562bec9e36a048b0a6d6952ffa5409ee', 'request_id': 'req-eb211bb6-8881-4a49-8135-422295dda8e8'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:33.637809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-eb211bb6-8881-4a49-8135-422295dda8e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '562bec9e36a048b0a6d6952ffa5409ee', 'request_id': 'req-eb211bb6-8881-4a49-8135-422295dda8e8'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:33.647900 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-eb211bb6-8881-4a49-8135-422295dda8e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '562bec9e36a048b0a6d6952ffa5409ee', 'request_id': 'req-eb211bb6-8881-4a49-8135-422295dda8e8'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:33.678164 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-eb211bb6-8881-4a49-8135-422295dda8e8 admin admin] The token cache is being invalidate because project 562bec9e36a048b0a6d6952ffa5409ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:27:33.678533 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-eb211bb6-8881-4a49-8135-422295dda8e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:33.679743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-eb211bb6-8881-4a49-8135-422295dda8e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:33.682133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1151/2292] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:27:33 2022] DELETE /identity/v3/projects/562bec9e36a048b0a6d6952ffa5409ee => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:27:33.686080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4f80b-9a7f-4e63-9a1d-387628f8de21 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:33.699964 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4f80b-9a7f-4e63-9a1d-387628f8de21 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:33.825818 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d5fea-cbf4-4a0f-af5e-9acbddc877ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:33.826668 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d5fea-cbf4-4a0f-af5e-9acbddc877ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:33.827444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d5fea-cbf4-4a0f-af5e-9acbddc877ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:33.827444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d5fea-cbf4-4a0f-af5e-9acbddc877ce admin admin] PATH_INFO: `/v3/users/54693b33c49b4a70860f208946027ef9` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:33.829445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d5fea-cbf4-4a0f-af5e-9acbddc877ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=54693b33c49b4a70860f208946027ef9)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:33.836111 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d5fea-cbf4-4a0f-af5e-9acbddc877ce admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:33.997766 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb211bb6-8881-4a49-8135-422295dda8e8 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:33.997766 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-eb211bb6-8881-4a49-8135-422295dda8e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:34.035477 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb211bb6-8881-4a49-8135-422295dda8e8 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:34.063365 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209c4c3-8eb9-43be-b439-a92f7bd91029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:34.064073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209c4c3-8eb9-43be-b439-a92f7bd91029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:34.066688 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209c4c3-8eb9-43be-b439-a92f7bd91029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:34.066977 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209c4c3-8eb9-43be-b439-a92f7bd91029 admin admin] PATH_INFO: `/v3/projects/b4dd02db0e39427f9a67d9d69443a8bc` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:34.069010 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209c4c3-8eb9-43be-b439-a92f7bd91029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4dd02db0e39427f9a67d9d69443a8bc)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:34.072893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209c4c3-8eb9-43be-b439-a92f7bd91029 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:34.098126 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-4d9d5fea-cbf4-4a0f-af5e-9acbddc877ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54693b33c49b4a70860f208946027ef9'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:34.140976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-4d9d5fea-cbf4-4a0f-af5e-9acbddc877ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54693b33c49b4a70860f208946027ef9'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:34.176963 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-4d9d5fea-cbf4-4a0f-af5e-9acbddc877ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54693b33c49b4a70860f208946027ef9'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:34.260314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-4d9d5fea-cbf4-4a0f-af5e-9acbddc877ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54693b33c49b4a70860f208946027ef9'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:34.262030 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-4d9d5fea-cbf4-4a0f-af5e-9acbddc877ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54693b33c49b4a70860f208946027ef9'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:34.263155 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d209c4c3-8eb9-43be-b439-a92f7bd91029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4dd02db0e39427f9a67d9d69443a8bc', 'request_id': 'req-d209c4c3-8eb9-43be-b439-a92f7bd91029'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:34.269929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1142/2293] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:27:33 2022] DELETE /identity/v3/users/54693b33c49b4a70860f208946027ef9 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:34.410294 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d209c4c3-8eb9-43be-b439-a92f7bd91029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4dd02db0e39427f9a67d9d69443a8bc', 'request_id': 'req-d209c4c3-8eb9-43be-b439-a92f7bd91029'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:34.424846 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d209c4c3-8eb9-43be-b439-a92f7bd91029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4dd02db0e39427f9a67d9d69443a8bc', 'request_id': 'req-d209c4c3-8eb9-43be-b439-a92f7bd91029'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:34.467737 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d209c4c3-8eb9-43be-b439-a92f7bd91029 admin admin] The token cache is being invalidate because project b4dd02db0e39427f9a67d9d69443a8bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:27:34.468329 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d209c4c3-8eb9-43be-b439-a92f7bd91029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:34.471884 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d209c4c3-8eb9-43be-b439-a92f7bd91029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:34.473665 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1152/2294] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:27:33 2022] DELETE /identity/v3/projects/b4dd02db0e39427f9a67d9d69443a8bc => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:27:35.280512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d5fea-cbf4-4a0f-af5e-9acbddc877ce admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:35.282762 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4d9d5fea-cbf4-4a0f-af5e-9acbddc877ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:35.332402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:35.469255 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d5fea-cbf4-4a0f-af5e-9acbddc877ce admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:35.483019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f72ee7-23c4-4dc2-8601-4e12e3f8e8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:35.484102 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f72ee7-23c4-4dc2-8601-4e12e3f8e8c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:35.484483 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f72ee7-23c4-4dc2-8601-4e12e3f8e8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:35.484859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f72ee7-23c4-4dc2-8601-4e12e3f8e8c9 admin admin] PATH_INFO: `/v3/projects/7ca18ee7c26c433fb8aa2c9496449c16` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:35.491051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f72ee7-23c4-4dc2-8601-4e12e3f8e8c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca18ee7c26c433fb8aa2c9496449c16)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:35.498605 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f72ee7-23c4-4dc2-8601-4e12e3f8e8c9 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:35.701655 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e6f72ee7-23c4-4dc2-8601-4e12e3f8e8c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca18ee7c26c433fb8aa2c9496449c16', 'request_id': 'req-e6f72ee7-23c4-4dc2-8601-4e12e3f8e8c9'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:35.714033 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e6f72ee7-23c4-4dc2-8601-4e12e3f8e8c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca18ee7c26c433fb8aa2c9496449c16', 'request_id': 'req-e6f72ee7-23c4-4dc2-8601-4e12e3f8e8c9'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:35.865975 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e6f72ee7-23c4-4dc2-8601-4e12e3f8e8c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca18ee7c26c433fb8aa2c9496449c16', 'request_id': 'req-e6f72ee7-23c4-4dc2-8601-4e12e3f8e8c9'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:35.886879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e6f72ee7-23c4-4dc2-8601-4e12e3f8e8c9 admin admin] The token cache is being invalidate because project 7ca18ee7c26c433fb8aa2c9496449c16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:27:35.889249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e6f72ee7-23c4-4dc2-8601-4e12e3f8e8c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:35.891587 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e6f72ee7-23c4-4dc2-8601-4e12e3f8e8c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:35.897334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1143/2295] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:27:35 2022] DELETE /identity/v3/projects/7ca18ee7c26c433fb8aa2c9496449c16 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:35.903235 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77589676-30ea-49c0-b9c4-ebe1a674619d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:35.904108 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77589676-30ea-49c0-b9c4-ebe1a674619d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:35.904682 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77589676-30ea-49c0-b9c4-ebe1a674619d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:35.990172 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-77589676-30ea-49c0-b9c4-ebe1a674619d None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:36.008047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-77589676-30ea-49c0-b9c4-ebe1a674619d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:36.173582 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1153/2296] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:36.182640 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb413f-4156-400d-bfca-34812f17ce5f None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:36.182912 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb413f-4156-400d-bfca-34812f17ce5f None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:36.183206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb413f-4156-400d-bfca-34812f17ce5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:36.229068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-f4cb413f-4156-400d-bfca-34812f17ce5f None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:36.232306 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f4cb413f-4156-400d-bfca-34812f17ce5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:36.255132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1144/2297] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:36.259541 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77589676-30ea-49c0-b9c4-ebe1a674619d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:36.275270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77589676-30ea-49c0-b9c4-ebe1a674619d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:36.296170 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd548a-b5dc-4f1a-aac2-9a0d9c978ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:36.297226 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd548a-b5dc-4f1a-aac2-9a0d9c978ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:36.298721 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd548a-b5dc-4f1a-aac2-9a0d9c978ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:36.299274 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd548a-b5dc-4f1a-aac2-9a0d9c978ce3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:36.306152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd548a-b5dc-4f1a-aac2-9a0d9c978ce3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:27:36.306152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd548a-b5dc-4f1a-aac2-9a0d9c978ce3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:36.306152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd548a-b5dc-4f1a-aac2-9a0d9c978ce3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:36.322135 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1154/2298] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:27:36.326144 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb413f-4156-400d-bfca-34812f17ce5f None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:36.338219 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb413f-4156-400d-bfca-34812f17ce5f None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:36.374676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da099e-c7f1-447a-bc20-4df2df3b14ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:36.376967 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da099e-c7f1-447a-bc20-4df2df3b14ee admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:36.377195 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da099e-c7f1-447a-bc20-4df2df3b14ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:36.377459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da099e-c7f1-447a-bc20-4df2df3b14ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:36.378131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da099e-c7f1-447a-bc20-4df2df3b14ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:36.380131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da099e-c7f1-447a-bc20-4df2df3b14ee admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:36.499255 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1145/2299] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:27:36 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:27:36.503346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd548a-b5dc-4f1a-aac2-9a0d9c978ce3 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:36.523924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd548a-b5dc-4f1a-aac2-9a0d9c978ce3 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:36.591217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:36.732033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94275f-f0e2-4674-8bcd-c269a1f5b714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:36.735504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94275f-f0e2-4674-8bcd-c269a1f5b714 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:36.735504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94275f-f0e2-4674-8bcd-c269a1f5b714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:36.735504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94275f-f0e2-4674-8bcd-c269a1f5b714 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:36.740224 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94275f-f0e2-4674-8bcd-c269a1f5b714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:36.743885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94275f-f0e2-4674-8bcd-c269a1f5b714 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:36.921426 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1155/2300] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:27:36 2022] POST /identity/v3/users => generated 292 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:27:36.925741 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da099e-c7f1-447a-bc20-4df2df3b14ee admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:36.954300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da099e-c7f1-447a-bc20-4df2df3b14ee admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:36.971397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef6ab2-ceda-4c5b-ac4a-ae3fbdfd3ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:36.972410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef6ab2-ceda-4c5b-ac4a-ae3fbdfd3ccc admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:36.973229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef6ab2-ceda-4c5b-ac4a-ae3fbdfd3ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:36.974499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef6ab2-ceda-4c5b-ac4a-ae3fbdfd3ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:36.975421 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef6ab2-ceda-4c5b-ac4a-ae3fbdfd3ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:36.978528 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef6ab2-ceda-4c5b-ac4a-ae3fbdfd3ccc admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:36.997407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1146/2301] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:27:37.001388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94275f-f0e2-4674-8bcd-c269a1f5b714 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:37.015424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94275f-f0e2-4674-8bcd-c269a1f5b714 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:37.041793 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d9db0-07bd-4c37-a57e-e25ef6d2eb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:37.042480 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d9db0-07bd-4c37-a57e-e25ef6d2eb96 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:37.042768 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d9db0-07bd-4c37-a57e-e25ef6d2eb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:37.043048 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d9db0-07bd-4c37-a57e-e25ef6d2eb96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:37.043747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d9db0-07bd-4c37-a57e-e25ef6d2eb96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:37.053318 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d9db0-07bd-4c37-a57e-e25ef6d2eb96 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:37.084284 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1156/2302] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:27:37.088539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef6ab2-ceda-4c5b-ac4a-ae3fbdfd3ccc admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:37.119077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef6ab2-ceda-4c5b-ac4a-ae3fbdfd3ccc admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:37.126820 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f130a-2027-4c64-9568-3829c41f0f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:37.127714 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f130a-2027-4c64-9568-3829c41f0f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:37.128137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f130a-2027-4c64-9568-3829c41f0f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:37.128546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f130a-2027-4c64-9568-3829c41f0f6d admin admin] PATH_INFO: `/v3/projects/e675e5ab57ce41388cc4d224f49ed0c3/users/9704e8ee2e4944f3a67b2dd76041360b/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:37.165125 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f130a-2027-4c64-9568-3829c41f0f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e675e5ab57ce41388cc4d224f49ed0c3, user_id=9704e8ee2e4944f3a67b2dd76041360b, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:37.169622 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f130a-2027-4c64-9568-3829c41f0f6d admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:37.268893 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1147/2303] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:27:37 2022] PUT /identity/v3/projects/e675e5ab57ce41388cc4d224f49ed0c3/users/9704e8ee2e4944f3a67b2dd76041360b/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:37.285417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff03ec-d8db-4e17-b31c-a9d1e4e819b7 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:37.285829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff03ec-d8db-4e17-b31c-a9d1e4e819b7 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:37.286313 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff03ec-d8db-4e17-b31c-a9d1e4e819b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:37.396240 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-52ff03ec-d8db-4e17-b31c-a9d1e4e819b7 None None] MFA Rules not processed for user `9704e8ee2e4944f3a67b2dd76041360b`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:37.398884 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-52ff03ec-d8db-4e17-b31c-a9d1e4e819b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:37.612449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1157/2304] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:37.613023 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f130a-2027-4c64-9568-3829c41f0f6d admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:37.815437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:37.863552 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f130a-2027-4c64-9568-3829c41f0f6d admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:37.884257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69858462-39aa-48fe-a9c5-dfb5564d9cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:37.884909 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69858462-39aa-48fe-a9c5-dfb5564d9cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:37.885316 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69858462-39aa-48fe-a9c5-dfb5564d9cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:37.885770 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69858462-39aa-48fe-a9c5-dfb5564d9cd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:37.887154 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69858462-39aa-48fe-a9c5-dfb5564d9cd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:37.890431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69858462-39aa-48fe-a9c5-dfb5564d9cd5 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:38.087298 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1148/2305] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:27:37 2022] POST /identity/v3/projects => generated 385 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:27:38.092044 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff03ec-d8db-4e17-b31c-a9d1e4e819b7 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:38.112719 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff03ec-d8db-4e17-b31c-a9d1e4e819b7 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:38.207533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad382db5-3d1e-421a-9b9e-d94f4de4941a None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:38.207912 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad382db5-3d1e-421a-9b9e-d94f4de4941a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:38.208266 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad382db5-3d1e-421a-9b9e-d94f4de4941a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:38.284192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-ad382db5-3d1e-421a-9b9e-d94f4de4941a None None] MFA Rules not processed for user `f9e2834bd72946aa8d6655d2937f7eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:38.285556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ad382db5-3d1e-421a-9b9e-d94f4de4941a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:38.325706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb9eb2-6127-42cc-a750-3efe12a6a1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:38.325706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb9eb2-6127-42cc-a750-3efe12a6a1f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:38.325706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb9eb2-6127-42cc-a750-3efe12a6a1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:38.325706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb9eb2-6127-42cc-a750-3efe12a6a1f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:38.326393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb9eb2-6127-42cc-a750-3efe12a6a1f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:38.328234 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb9eb2-6127-42cc-a750-3efe12a6a1f7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:38.646899 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1149/2306] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:27:38.661583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad382db5-3d1e-421a-9b9e-d94f4de4941a None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:38.663693 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ad382db5-3d1e-421a-9b9e-d94f4de4941a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:38.748484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1158/2307] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:27:38 2022] POST /identity/v3/users => generated 293 bytes in 657 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:27:38.752416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb9eb2-6127-42cc-a750-3efe12a6a1f7 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:38.777789 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb9eb2-6127-42cc-a750-3efe12a6a1f7 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:38.801250 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5905787-b067-4f96-956a-7f3cd6e81fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:38.802258 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5905787-b067-4f96-956a-7f3cd6e81fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:38.802258 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5905787-b067-4f96-956a-7f3cd6e81fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:38.802471 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5905787-b067-4f96-956a-7f3cd6e81fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:38.803079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5905787-b067-4f96-956a-7f3cd6e81fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:38.811010 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5905787-b067-4f96-956a-7f3cd6e81fd1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:38.818097 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad382db5-3d1e-421a-9b9e-d94f4de4941a None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:38.823897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea123821-4ed8-4ba8-82d8-e283f95622f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:38.824986 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea123821-4ed8-4ba8-82d8-e283f95622f5 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:38.825496 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea123821-4ed8-4ba8-82d8-e283f95622f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:38.826482 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea123821-4ed8-4ba8-82d8-e283f95622f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:38.828960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ea123821-4ed8-4ba8-82d8-e283f95622f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:38.839315 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1159/2308] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:27:38.844284 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5905787-b067-4f96-956a-7f3cd6e81fd1 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:38.912737 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5905787-b067-4f96-956a-7f3cd6e81fd1 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:38.934619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285de10d-6f81-4abb-8667-e9d924452c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:38.935117 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285de10d-6f81-4abb-8667-e9d924452c98 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:38.937713 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285de10d-6f81-4abb-8667-e9d924452c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:38.938007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285de10d-6f81-4abb-8667-e9d924452c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:38.938702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285de10d-6f81-4abb-8667-e9d924452c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:38.940381 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285de10d-6f81-4abb-8667-e9d924452c98 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:38.964717 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:38.994122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1160/2309] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:27:39.006311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3190e6-fd81-4ff3-8f77-534ed023de80 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:39.006311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3190e6-fd81-4ff3-8f77-534ed023de80 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:39.006311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3190e6-fd81-4ff3-8f77-534ed023de80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:39.024101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea123821-4ed8-4ba8-82d8-e283f95622f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:39.027654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea123821-4ed8-4ba8-82d8-e283f95622f5 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:39.041844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1150/2310] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:27:39.042977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea123821-4ed8-4ba8-82d8-e283f95622f5 service nova] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:39.078920 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea123821-4ed8-4ba8-82d8-e283f95622f5 service nova] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:39.084604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9df5a-0595-44b8-bde1-a2d863f2094f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:39.085975 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9df5a-0595-44b8-bde1-a2d863f2094f admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:39.086359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9df5a-0595-44b8-bde1-a2d863f2094f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:39.086774 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9df5a-0595-44b8-bde1-a2d863f2094f admin admin] PATH_INFO: `/v3/projects/0ec88c6892ef4fe09936f2af14bb7003/users/ca8d4d99e6034fdea963c37e1717c3d7/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:39.098081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-fe3190e6-fd81-4ff3-8f77-534ed023de80 None None] MFA Rules not processed for user `5a667fa07b1148998959835d9995ad60`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:39.103434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:39.108014 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fe3190e6-fd81-4ff3-8f77-534ed023de80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:39.117902 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9df5a-0595-44b8-bde1-a2d863f2094f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec88c6892ef4fe09936f2af14bb7003, user_id=ca8d4d99e6034fdea963c37e1717c3d7, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:39.126861 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9df5a-0595-44b8-bde1-a2d863f2094f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:39.220657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1151/2311] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:27:39 2022] PUT /identity/v3/projects/0ec88c6892ef4fe09936f2af14bb7003/users/ca8d4d99e6034fdea963c37e1717c3d7/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:39.232899 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c38f7-1794-4fb9-a968-1f3d26a16c98 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:39.233324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c38f7-1794-4fb9-a968-1f3d26a16c98 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:39.233773 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c38f7-1794-4fb9-a968-1f3d26a16c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:39.265061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1161/2312] 10.222.0.78 () {58 vars in 1009 bytes} [Mon Sep 19 11:27:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:39.289464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3190e6-fd81-4ff3-8f77-534ed023de80 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:39.291908 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fe3190e6-fd81-4ff3-8f77-534ed023de80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:39.350652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-446c38f7-1794-4fb9-a968-1f3d26a16c98 None None] MFA Rules not processed for user `ca8d4d99e6034fdea963c37e1717c3d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:39.356932 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-446c38f7-1794-4fb9-a968-1f3d26a16c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:39.551033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3190e6-fd81-4ff3-8f77-534ed023de80 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:39.575660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1152/2313] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:27:39.577808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c38f7-1794-4fb9-a968-1f3d26a16c98 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:39.729652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d966633-2eb2-4211-982d-4377db31c1da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf462b2aa3fc4e2ca58851549cf0c644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:39.734766 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d966633-2eb2-4211-982d-4377db31c1da service swift] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:39.735119 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d966633-2eb2-4211-982d-4377db31c1da service swift] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:39.735398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d966633-2eb2-4211-982d-4377db31c1da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:39.786514 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c38f7-1794-4fb9-a968-1f3d26a16c98 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:39.796338 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbeebc-7c92-4e12-8361-8fb279c23098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:39.797567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbeebc-7c92-4e12-8361-8fb279c23098 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:39.797848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbeebc-7c92-4e12-8361-8fb279c23098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:39.798121 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbeebc-7c92-4e12-8361-8fb279c23098 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:39.802944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbeebc-7c92-4e12-8361-8fb279c23098 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:39.807455 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbeebc-7c92-4e12-8361-8fb279c23098 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:39.885636 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:39.911492 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d966633-2eb2-4211-982d-4377db31c1da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:39.913329 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d966633-2eb2-4211-982d-4377db31c1da service swift] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:39.934530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1162/2314] 10.222.0.78 () {60 vars in 1384 bytes} [Mon Sep 19 11:27:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 11:27:39.954039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d966633-2eb2-4211-982d-4377db31c1da service swift] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:39.954717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1153/2315] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:27:39 2022] POST /identity/v3/projects => generated 383 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:27:40.008073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d966633-2eb2-4211-982d-4377db31c1da service swift] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:40.182564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944a052-01f1-46b6-91b2-c646ae0a0de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:40.188042 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944a052-01f1-46b6-91b2-c646ae0a0de0 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:40.191044 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944a052-01f1-46b6-91b2-c646ae0a0de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:40.199976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944a052-01f1-46b6-91b2-c646ae0a0de0 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:40.201544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944a052-01f1-46b6-91b2-c646ae0a0de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:40.204171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944a052-01f1-46b6-91b2-c646ae0a0de0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:40.313033 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbeebc-7c92-4e12-8361-8fb279c23098 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:40.314306 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1163/2316] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:27:39 2022] POST /identity/v3/users => generated 298 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:27:40.335224 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbeebc-7c92-4e12-8361-8fb279c23098 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:40.344448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ccc6b2-405d-428e-8ff8-7f97dcf996b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:40.348334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccc6b2-405d-428e-8ff8-7f97dcf996b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:40.357297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccc6b2-405d-428e-8ff8-7f97dcf996b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:40.357835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccc6b2-405d-428e-8ff8-7f97dcf996b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:40.358936 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ccc6b2-405d-428e-8ff8-7f97dcf996b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:40.360999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ccc6b2-405d-428e-8ff8-7f97dcf996b6 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:40.428272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1154/2317] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:27:40.432587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944a052-01f1-46b6-91b2-c646ae0a0de0 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:40.447460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944a052-01f1-46b6-91b2-c646ae0a0de0 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:40.469710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9b94f-7499-4fdb-9338-1665fdd1e1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:40.475689 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9b94f-7499-4fdb-9338-1665fdd1e1df admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:40.475689 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9b94f-7499-4fdb-9338-1665fdd1e1df admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:40.475689 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9b94f-7499-4fdb-9338-1665fdd1e1df admin admin] PATH_INFO: `/v3/projects/3965582aad3c44ec8063ccedadf11b3d/users/04bdca23ac70461194c237d1769492a4/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:40.515234 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9b94f-7499-4fdb-9338-1665fdd1e1df admin admin] RBAC: Authorizing `identity:create_grant(project_id=3965582aad3c44ec8063ccedadf11b3d, user_id=04bdca23ac70461194c237d1769492a4, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:40.520168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9b94f-7499-4fdb-9338-1665fdd1e1df admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:40.613528 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1164/2318] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:27:40 2022] PUT /identity/v3/projects/3965582aad3c44ec8063ccedadf11b3d/users/04bdca23ac70461194c237d1769492a4/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:40.615585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ccc6b2-405d-428e-8ff8-7f97dcf996b6 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:40.644636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ccc6b2-405d-428e-8ff8-7f97dcf996b6 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:40.761319 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:40.827662 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316eb38d-d5e4-4bcc-b5dc-b39478cf6ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:40.828330 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316eb38d-d5e4-4bcc-b5dc-b39478cf6ade admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:40.828733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316eb38d-d5e4-4bcc-b5dc-b39478cf6ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:40.829532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316eb38d-d5e4-4bcc-b5dc-b39478cf6ade admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:40.830304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316eb38d-d5e4-4bcc-b5dc-b39478cf6ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:40.832105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316eb38d-d5e4-4bcc-b5dc-b39478cf6ade admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:40.867082 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1155/2319] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:27:40.877505 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9b94f-7499-4fdb-9338-1665fdd1e1df admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:40.896528 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9b94f-7499-4fdb-9338-1665fdd1e1df admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:40.919045 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3389b1-c291-4454-b292-e714ebb475ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:40.919916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3389b1-c291-4454-b292-e714ebb475ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:40.920649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3389b1-c291-4454-b292-e714ebb475ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:40.921667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3389b1-c291-4454-b292-e714ebb475ea admin admin] PATH_INFO: `/v3/projects/3965582aad3c44ec8063ccedadf11b3d/users/04bdca23ac70461194c237d1769492a4/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:40.925882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3389b1-c291-4454-b292-e714ebb475ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=3965582aad3c44ec8063ccedadf11b3d, user_id=04bdca23ac70461194c237d1769492a4, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:40.936970 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3389b1-c291-4454-b292-e714ebb475ea admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:40.974373 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1165/2320] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:27:40 2022] PUT /identity/v3/projects/3965582aad3c44ec8063ccedadf11b3d/users/04bdca23ac70461194c237d1769492a4/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:40.986937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690bcf4-2aa2-4cfd-90c6-2ef8da7f9db2 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:40.988306 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690bcf4-2aa2-4cfd-90c6-2ef8da7f9db2 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:40.988306 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690bcf4-2aa2-4cfd-90c6-2ef8da7f9db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:41.092976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-7690bcf4-2aa2-4cfd-90c6-2ef8da7f9db2 None None] MFA Rules not processed for user `04bdca23ac70461194c237d1769492a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:41.095698 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7690bcf4-2aa2-4cfd-90c6-2ef8da7f9db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:41.370045 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3389b1-c291-4454-b292-e714ebb475ea admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:41.372285 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4b3389b1-c291-4454-b292-e714ebb475ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:41.380199 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1156/2321] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:41.557534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3389b1-c291-4454-b292-e714ebb475ea admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:41.572062 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43275338-538d-434b-912c-df26075b4081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:41.572600 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43275338-538d-434b-912c-df26075b4081 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:41.572908 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43275338-538d-434b-912c-df26075b4081 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:41.574206 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43275338-538d-434b-912c-df26075b4081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:41.725475 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:41.734822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43275338-538d-434b-912c-df26075b4081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:41.743874 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43275338-538d-434b-912c-df26075b4081 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:41.797837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1166/2322] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:27:44.475127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690bcf4-2aa2-4cfd-90c6-2ef8da7f9db2 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:44.476765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7690bcf4-2aa2-4cfd-90c6-2ef8da7f9db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:44.489548 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690bcf4-2aa2-4cfd-90c6-2ef8da7f9db2 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:44.493506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e619e-588e-4dda-b99d-ad600da92cee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:44.494069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e619e-588e-4dda-b99d-ad600da92cee service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:44.494342 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e619e-588e-4dda-b99d-ad600da92cee service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:44.494618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e619e-588e-4dda-b99d-ad600da92cee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:44.578991 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e619e-588e-4dda-b99d-ad600da92cee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:44.580524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e619e-588e-4dda-b99d-ad600da92cee service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:44.615434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1157/2323] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:27:45.369074 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cf744-85a8-4c71-a108-9f6092c311d6 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:45.375146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cf744-85a8-4c71-a108-9f6092c311d6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:45.375712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cf744-85a8-4c71-a108-9f6092c311d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:45.437012 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a53cf744-85a8-4c71-a108-9f6092c311d6 None None] MFA Rules not processed for user `792178b0829f49c2b5729aeee1805e77`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:45.448315 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a53cf744-85a8-4c71-a108-9f6092c311d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:45.603007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1167/2324] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:46.936010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e619e-588e-4dda-b99d-ad600da92cee service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:46.959682 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e619e-588e-4dda-b99d-ad600da92cee service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:46.973520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f57f8f-adb9-417a-9a13-cca8a1c61846 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:46.978857 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f57f8f-adb9-417a-9a13-cca8a1c61846 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:46.979482 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f57f8f-adb9-417a-9a13-cca8a1c61846 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:46.982362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f57f8f-adb9-417a-9a13-cca8a1c61846 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:47.020575 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f57f8f-adb9-417a-9a13-cca8a1c61846 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:47.026281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f57f8f-adb9-417a-9a13-cca8a1c61846 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:47.059158 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1158/2325] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:27:53.923570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cf744-85a8-4c71-a108-9f6092c311d6 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:53.999457 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cf744-85a8-4c71-a108-9f6092c311d6 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:54.004891 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8067f91-a8fc-4ddd-9fab-7fe75532d846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:54.005761 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8067f91-a8fc-4ddd-9fab-7fe75532d846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:54.006235 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8067f91-a8fc-4ddd-9fab-7fe75532d846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:54.006751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8067f91-a8fc-4ddd-9fab-7fe75532d846 admin admin] PATH_INFO: `/v3/users/9704e8ee2e4944f3a67b2dd76041360b` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:54.008734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8067f91-a8fc-4ddd-9fab-7fe75532d846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9704e8ee2e4944f3a67b2dd76041360b)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:54.010689 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8067f91-a8fc-4ddd-9fab-7fe75532d846 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:54.208716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b8067f91-a8fc-4ddd-9fab-7fe75532d846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9704e8ee2e4944f3a67b2dd76041360b'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:54.231488 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b8067f91-a8fc-4ddd-9fab-7fe75532d846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9704e8ee2e4944f3a67b2dd76041360b'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:54.241884 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b8067f91-a8fc-4ddd-9fab-7fe75532d846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9704e8ee2e4944f3a67b2dd76041360b'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:54.340415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b8067f91-a8fc-4ddd-9fab-7fe75532d846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9704e8ee2e4944f3a67b2dd76041360b'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:54.341404 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b8067f91-a8fc-4ddd-9fab-7fe75532d846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9704e8ee2e4944f3a67b2dd76041360b'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:54.350886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1168/2326] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:27:53 2022] DELETE /identity/v3/users/9704e8ee2e4944f3a67b2dd76041360b => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:54.356745 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56117ef1-907e-4dcb-a81a-5fda0030e3e3 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:54.358719 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56117ef1-907e-4dcb-a81a-5fda0030e3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:54.358719 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56117ef1-907e-4dcb-a81a-5fda0030e3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:54.384826 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-56117ef1-907e-4dcb-a81a-5fda0030e3e3 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:54.386677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-56117ef1-907e-4dcb-a81a-5fda0030e3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:54.432858 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:54.471036 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1159/2327] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:54.483109 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8067f91-a8fc-4ddd-9fab-7fe75532d846 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:54.485296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b8067f91-a8fc-4ddd-9fab-7fe75532d846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:54.526333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:54.587460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8067f91-a8fc-4ddd-9fab-7fe75532d846 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:54.592868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ceb56-b003-4550-b2ec-143ef5511a8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:54.594295 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ceb56-b003-4550-b2ec-143ef5511a8e service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:54.594818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ceb56-b003-4550-b2ec-143ef5511a8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:54.595413 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ceb56-b003-4550-b2ec-143ef5511a8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:54.609123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ceb56-b003-4550-b2ec-143ef5511a8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:54.610864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ceb56-b003-4550-b2ec-143ef5511a8e service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:54.624392 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1169/2328] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:27:55.239411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56117ef1-907e-4dcb-a81a-5fda0030e3e3 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:55.241794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-56117ef1-907e-4dcb-a81a-5fda0030e3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:55.274970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56117ef1-907e-4dcb-a81a-5fda0030e3e3 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:55.291151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe36b2c-6530-4882-83de-26407fc7c739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:55.292157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe36b2c-6530-4882-83de-26407fc7c739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:55.295561 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe36b2c-6530-4882-83de-26407fc7c739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:55.296013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe36b2c-6530-4882-83de-26407fc7c739 admin admin] PATH_INFO: `/v3/projects/e675e5ab57ce41388cc4d224f49ed0c3` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:55.299785 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe36b2c-6530-4882-83de-26407fc7c739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e675e5ab57ce41388cc4d224f49ed0c3)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:55.302159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe36b2c-6530-4882-83de-26407fc7c739 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:55.500449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-dfe36b2c-6530-4882-83de-26407fc7c739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e675e5ab57ce41388cc4d224f49ed0c3', 'request_id': 'req-dfe36b2c-6530-4882-83de-26407fc7c739'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:55.506539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-dfe36b2c-6530-4882-83de-26407fc7c739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e675e5ab57ce41388cc4d224f49ed0c3', 'request_id': 'req-dfe36b2c-6530-4882-83de-26407fc7c739'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:55.612659 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-dfe36b2c-6530-4882-83de-26407fc7c739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e675e5ab57ce41388cc4d224f49ed0c3', 'request_id': 'req-dfe36b2c-6530-4882-83de-26407fc7c739'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:55.624695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-dfe36b2c-6530-4882-83de-26407fc7c739 admin admin] The token cache is being invalidate because project e675e5ab57ce41388cc4d224f49ed0c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:27:55.624984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-dfe36b2c-6530-4882-83de-26407fc7c739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:55.625655 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-dfe36b2c-6530-4882-83de-26407fc7c739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:55.627156 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1160/2329] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:27:55 2022] DELETE /identity/v3/projects/e675e5ab57ce41388cc4d224f49ed0c3 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:55.631235 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ceb56-b003-4550-b2ec-143ef5511a8e service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:55.641374 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ceb56-b003-4550-b2ec-143ef5511a8e service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:55.697534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdcd95-c6e8-4c8c-9dcd-714a74d76101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:55.698162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdcd95-c6e8-4c8c-9dcd-714a74d76101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:55.698440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdcd95-c6e8-4c8c-9dcd-714a74d76101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:55.698706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdcd95-c6e8-4c8c-9dcd-714a74d76101 admin admin] PATH_INFO: `/v3/users/ca8d4d99e6034fdea963c37e1717c3d7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:55.699837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdcd95-c6e8-4c8c-9dcd-714a74d76101 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca8d4d99e6034fdea963c37e1717c3d7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:55.701791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdcd95-c6e8-4c8c-9dcd-714a74d76101 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:55.982407 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e5cdcd95-c6e8-4c8c-9dcd-714a74d76101 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca8d4d99e6034fdea963c37e1717c3d7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:55.995206 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e5cdcd95-c6e8-4c8c-9dcd-714a74d76101 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca8d4d99e6034fdea963c37e1717c3d7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:56.002162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e5cdcd95-c6e8-4c8c-9dcd-714a74d76101 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca8d4d99e6034fdea963c37e1717c3d7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:56.081693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e5cdcd95-c6e8-4c8c-9dcd-714a74d76101 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca8d4d99e6034fdea963c37e1717c3d7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:56.086102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e5cdcd95-c6e8-4c8c-9dcd-714a74d76101 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca8d4d99e6034fdea963c37e1717c3d7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:56.092247 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1170/2330] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:27:55 2022] DELETE /identity/v3/users/ca8d4d99e6034fdea963c37e1717c3d7 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:56.480050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb70af-24b2-4e96-a137-97bb9156d579 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:56.480760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb70af-24b2-4e96-a137-97bb9156d579 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:56.481037 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb70af-24b2-4e96-a137-97bb9156d579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:56.501878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-98cb70af-24b2-4e96-a137-97bb9156d579 None None] MFA Rules not processed for user `5a667fa07b1148998959835d9995ad60`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:56.503183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-98cb70af-24b2-4e96-a137-97bb9156d579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:56.560115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1161/2331] 10.222.0.78 () {58 vars in 967 bytes} [Mon Sep 19 11:27:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:56.573115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdcd95-c6e8-4c8c-9dcd-714a74d76101 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:56.575264 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e5cdcd95-c6e8-4c8c-9dcd-714a74d76101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:56.625957 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:56.637942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb70af-24b2-4e96-a137-97bb9156d579 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:56.639521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-98cb70af-24b2-4e96-a137-97bb9156d579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:56.666629 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdcd95-c6e8-4c8c-9dcd-714a74d76101 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:56.685562 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b13712-d275-4471-a6f1-55cf1678050b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf462b2aa3fc4e2ca58851549cf0c644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:56.686628 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b13712-d275-4471-a6f1-55cf1678050b service swift] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:56.686925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b13712-d275-4471-a6f1-55cf1678050b service swift] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:56.687364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b13712-d275-4471-a6f1-55cf1678050b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:56.706957 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b13712-d275-4471-a6f1-55cf1678050b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:56.706957 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b13712-d275-4471-a6f1-55cf1678050b service swift] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:56.718911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1171/2332] 10.222.0.78 () {60 vars in 1384 bytes} [Mon Sep 19 11:27:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:27:56.746440 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb70af-24b2-4e96-a137-97bb9156d579 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:56.751719 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16de322-034d-42b1-b3f6-8df134853aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:56.752382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16de322-034d-42b1-b3f6-8df134853aca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:56.752845 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16de322-034d-42b1-b3f6-8df134853aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:56.753312 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16de322-034d-42b1-b3f6-8df134853aca admin admin] PATH_INFO: `/v3/projects/0ec88c6892ef4fe09936f2af14bb7003` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:56.755947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16de322-034d-42b1-b3f6-8df134853aca admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ec88c6892ef4fe09936f2af14bb7003)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:56.760965 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16de322-034d-42b1-b3f6-8df134853aca admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:56.778364 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:56.899640 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a16de322-034d-42b1-b3f6-8df134853aca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ec88c6892ef4fe09936f2af14bb7003', 'request_id': 'req-a16de322-034d-42b1-b3f6-8df134853aca'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:56.906251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a16de322-034d-42b1-b3f6-8df134853aca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ec88c6892ef4fe09936f2af14bb7003', 'request_id': 'req-a16de322-034d-42b1-b3f6-8df134853aca'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:57.037584 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a16de322-034d-42b1-b3f6-8df134853aca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ec88c6892ef4fe09936f2af14bb7003', 'request_id': 'req-a16de322-034d-42b1-b3f6-8df134853aca'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:57.051331 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a16de322-034d-42b1-b3f6-8df134853aca admin admin] The token cache is being invalidate because project 0ec88c6892ef4fe09936f2af14bb7003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:27:57.051628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a16de322-034d-42b1-b3f6-8df134853aca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:57.052330 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a16de322-034d-42b1-b3f6-8df134853aca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:57.053876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1162/2333] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:27:56 2022] DELETE /identity/v3/projects/0ec88c6892ef4fe09936f2af14bb7003 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:57.059350 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b13712-d275-4471-a6f1-55cf1678050b service swift] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:57.070176 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b13712-d275-4471-a6f1-55cf1678050b service swift] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:57.139391 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7ba5d-0df3-4b0b-8495-d7358e22f415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:57.140128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7ba5d-0df3-4b0b-8495-d7358e22f415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:57.140456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7ba5d-0df3-4b0b-8495-d7358e22f415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:57.140738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7ba5d-0df3-4b0b-8495-d7358e22f415 admin admin] PATH_INFO: `/v3/users/04bdca23ac70461194c237d1769492a4` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:57.141918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7ba5d-0df3-4b0b-8495-d7358e22f415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04bdca23ac70461194c237d1769492a4)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:57.143881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7ba5d-0df3-4b0b-8495-d7358e22f415 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:57.349241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-68f7ba5d-0df3-4b0b-8495-d7358e22f415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04bdca23ac70461194c237d1769492a4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:57.362240 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-68f7ba5d-0df3-4b0b-8495-d7358e22f415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04bdca23ac70461194c237d1769492a4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:57.369371 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-68f7ba5d-0df3-4b0b-8495-d7358e22f415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04bdca23ac70461194c237d1769492a4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:57.453297 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-68f7ba5d-0df3-4b0b-8495-d7358e22f415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04bdca23ac70461194c237d1769492a4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:57.454145 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-68f7ba5d-0df3-4b0b-8495-d7358e22f415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04bdca23ac70461194c237d1769492a4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:57.464581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1172/2334] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:27:57 2022] DELETE /identity/v3/users/04bdca23ac70461194c237d1769492a4 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:57.919521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16de322-034d-42b1-b3f6-8df134853aca admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:57.921313 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a16de322-034d-42b1-b3f6-8df134853aca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:57.939942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16de322-034d-42b1-b3f6-8df134853aca admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:58.011389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ea987-1fc7-45bd-ab6b-9936edcc56f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:58.012029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ea987-1fc7-45bd-ab6b-9936edcc56f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:58.012309 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ea987-1fc7-45bd-ab6b-9936edcc56f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:58.012661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ea987-1fc7-45bd-ab6b-9936edcc56f8 admin admin] PATH_INFO: `/v3/projects/3965582aad3c44ec8063ccedadf11b3d` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:58.014028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ea987-1fc7-45bd-ab6b-9936edcc56f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3965582aad3c44ec8063ccedadf11b3d)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:58.016069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ea987-1fc7-45bd-ab6b-9936edcc56f8 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:58.153780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-cc9ea987-1fc7-45bd-ab6b-9936edcc56f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3965582aad3c44ec8063ccedadf11b3d', 'request_id': 'req-cc9ea987-1fc7-45bd-ab6b-9936edcc56f8'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:58.161971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-cc9ea987-1fc7-45bd-ab6b-9936edcc56f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3965582aad3c44ec8063ccedadf11b3d', 'request_id': 'req-cc9ea987-1fc7-45bd-ab6b-9936edcc56f8'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:58.261315 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-cc9ea987-1fc7-45bd-ab6b-9936edcc56f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3965582aad3c44ec8063ccedadf11b3d', 'request_id': 'req-cc9ea987-1fc7-45bd-ab6b-9936edcc56f8'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:58.274642 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-cc9ea987-1fc7-45bd-ab6b-9936edcc56f8 admin admin] The token cache is being invalidate because project 3965582aad3c44ec8063ccedadf11b3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:27:58.274936 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-cc9ea987-1fc7-45bd-ab6b-9936edcc56f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:58.275636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-cc9ea987-1fc7-45bd-ab6b-9936edcc56f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:27:58.277253 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1163/2335] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:27:57 2022] DELETE /identity/v3/projects/3965582aad3c44ec8063ccedadf11b3d => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:58.282901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc784e3-5bd2-4337-b525-f238305fe5af None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:58.283591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc784e3-5bd2-4337-b525-f238305fe5af None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:58.283591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc784e3-5bd2-4337-b525-f238305fe5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:58.304276 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0cc784e3-5bd2-4337-b525-f238305fe5af None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:58.306110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0cc784e3-5bd2-4337-b525-f238305fe5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:58.356346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:58.376878 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1173/2336] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:58.392938 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c4c30-faf7-4f4c-bf82-332f3bab2071 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:58.394138 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c4c30-faf7-4f4c-bf82-332f3bab2071 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:58.394477 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c4c30-faf7-4f4c-bf82-332f3bab2071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:58.420015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-0f8c4c30-faf7-4f4c-bf82-332f3bab2071 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:58.421173 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-0f8c4c30-faf7-4f4c-bf82-332f3bab2071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:58.433655 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1164/2337] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:58.438767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc784e3-5bd2-4337-b525-f238305fe5af None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:58.452843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc784e3-5bd2-4337-b525-f238305fe5af None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:58.458652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b690565-bc74-4b83-ba7f-8692c31c1469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:58.459452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b690565-bc74-4b83-ba7f-8692c31c1469 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:58.459904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b690565-bc74-4b83-ba7f-8692c31c1469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:58.460313 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b690565-bc74-4b83-ba7f-8692c31c1469 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:58.461143 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b690565-bc74-4b83-ba7f-8692c31c1469 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:27:58.461809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b690565-bc74-4b83-ba7f-8692c31c1469 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:58.463691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b690565-bc74-4b83-ba7f-8692c31c1469 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:58.476544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1174/2338] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:27:58.481632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c4c30-faf7-4f4c-bf82-332f3bab2071 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:58.491482 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c4c30-faf7-4f4c-bf82-332f3bab2071 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:58.496871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a887d-0dc4-4196-a321-85a08f6031b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:58.497491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a887d-0dc4-4196-a321-85a08f6031b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:58.497768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a887d-0dc4-4196-a321-85a08f6031b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:58.498043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a887d-0dc4-4196-a321-85a08f6031b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:58.498679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a887d-0dc4-4196-a321-85a08f6031b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:58.500676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a887d-0dc4-4196-a321-85a08f6031b6 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:58.605815 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:58.612532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1165/2339] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:27:58 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:27:58.617428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b690565-bc74-4b83-ba7f-8692c31c1469 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:58.634628 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b690565-bc74-4b83-ba7f-8692c31c1469 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:58.722907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766ea7e-5088-4638-a4de-ad64336d93ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:58.723591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766ea7e-5088-4638-a4de-ad64336d93ed admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:58.723874 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766ea7e-5088-4638-a4de-ad64336d93ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:58.724150 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766ea7e-5088-4638-a4de-ad64336d93ed admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:58.724945 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766ea7e-5088-4638-a4de-ad64336d93ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:58.726959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766ea7e-5088-4638-a4de-ad64336d93ed admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:58.901436 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1175/2340] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:27:58 2022] POST /identity/v3/users => generated 293 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:27:58.905691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a887d-0dc4-4196-a321-85a08f6031b6 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:58.918306 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a887d-0dc4-4196-a321-85a08f6031b6 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:58.927061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef814d3-8535-4ae8-9213-12b78bd3cd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:58.927846 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef814d3-8535-4ae8-9213-12b78bd3cd13 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:58.927978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef814d3-8535-4ae8-9213-12b78bd3cd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:58.928250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef814d3-8535-4ae8-9213-12b78bd3cd13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:58.928912 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef814d3-8535-4ae8-9213-12b78bd3cd13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:58.930726 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef814d3-8535-4ae8-9213-12b78bd3cd13 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:58.942344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1166/2341] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:27:58.946475 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766ea7e-5088-4638-a4de-ad64336d93ed admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:58.956876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766ea7e-5088-4638-a4de-ad64336d93ed admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:58.962334 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14adfe-fccf-45e1-af04-1e11367fc772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:58.962940 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14adfe-fccf-45e1-af04-1e11367fc772 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:58.963253 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14adfe-fccf-45e1-af04-1e11367fc772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:58.963534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14adfe-fccf-45e1-af04-1e11367fc772 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:58.964230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14adfe-fccf-45e1-af04-1e11367fc772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:58.965786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14adfe-fccf-45e1-af04-1e11367fc772 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:58.979546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1176/2342] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:27:58.987082 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef814d3-8535-4ae8-9213-12b78bd3cd13 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:58.998097 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef814d3-8535-4ae8-9213-12b78bd3cd13 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:59.012275 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971cae4-9247-4542-aefb-1e838a15a5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:59.013040 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971cae4-9247-4542-aefb-1e838a15a5fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:59.013339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971cae4-9247-4542-aefb-1e838a15a5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:59.015472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971cae4-9247-4542-aefb-1e838a15a5fe admin admin] PATH_INFO: `/v3/projects/5facd1db989f4e21aedcfa7cfaca8eaa/users/54a6fee7082347e4957459e7ade47ece/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:59.041673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7971cae4-9247-4542-aefb-1e838a15a5fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5facd1db989f4e21aedcfa7cfaca8eaa, user_id=54a6fee7082347e4957459e7ade47ece, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:59.044565 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7971cae4-9247-4542-aefb-1e838a15a5fe admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:59.179099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1167/2343] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:27:58 2022] PUT /identity/v3/projects/5facd1db989f4e21aedcfa7cfaca8eaa/users/54a6fee7082347e4957459e7ade47ece/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:27:59.190276 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2d069-6c91-4301-9ee3-b95cb0b5e3f0 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:59.190667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2d069-6c91-4301-9ee3-b95cb0b5e3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:59.191034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2d069-6c91-4301-9ee3-b95cb0b5e3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:59.234852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-9ca2d069-6c91-4301-9ee3-b95cb0b5e3f0 None None] MFA Rules not processed for user `54a6fee7082347e4957459e7ade47ece`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:59.236043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9ca2d069-6c91-4301-9ee3-b95cb0b5e3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:59.335252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1177/2344] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:27:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:59.348960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971cae4-9247-4542-aefb-1e838a15a5fe admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:59.350965 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7971cae4-9247-4542-aefb-1e838a15a5fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:59.413242 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971cae4-9247-4542-aefb-1e838a15a5fe admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:59.417949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe381b6-b11d-41c9-acf5-0384cd2b53ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:59.418628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe381b6-b11d-41c9-acf5-0384cd2b53ed service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:59.418816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe381b6-b11d-41c9-acf5-0384cd2b53ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:59.419140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe381b6-b11d-41c9-acf5-0384cd2b53ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:59.430426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe381b6-b11d-41c9-acf5-0384cd2b53ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:59.432984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe381b6-b11d-41c9-acf5-0384cd2b53ed service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:59.443647 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1168/2345] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:28:00.206100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3edca3-1aaa-4071-8bf3-905a159f14c1 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:00.206410 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3edca3-1aaa-4071-8bf3-905a159f14c1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:00.206684 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3edca3-1aaa-4071-8bf3-905a159f14c1 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:00.208083 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1178/2346] 10.222.0.78 () {54 vars in 961 bytes} [Mon Sep 19 11:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:28:00.214198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15465968-88d5-479c-bcb8-70ed07dc6e58 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:00.214965 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15465968-88d5-479c-bcb8-70ed07dc6e58 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:00.214965 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15465968-88d5-479c-bcb8-70ed07dc6e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:00.239695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-15465968-88d5-479c-bcb8-70ed07dc6e58 None None] MFA Rules not processed for user `0427cdd3905a422cab4fed0aac187ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:00.240957 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-15465968-88d5-479c-bcb8-70ed07dc6e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:00.377069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1169/2347] 10.222.0.78 () {58 vars in 1058 bytes} [Mon Sep 19 11:28:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:00.391211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3edca3-1aaa-4071-8bf3-905a159f14c1 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:00.402361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3edca3-1aaa-4071-8bf3-905a159f14c1 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:00.407807 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44931be-69d6-4e8e-9a2c-1c4416fd27b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:00.408435 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44931be-69d6-4e8e-9a2c-1c4416fd27b6 service placement] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:00.408720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44931be-69d6-4e8e-9a2c-1c4416fd27b6 service placement] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:00.408995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44931be-69d6-4e8e-9a2c-1c4416fd27b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:00.410804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f44931be-69d6-4e8e-9a2c-1c4416fd27b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:00.430451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44931be-69d6-4e8e-9a2c-1c4416fd27b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:00.433726 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15465968-88d5-479c-bcb8-70ed07dc6e58 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:00.437355 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-15465968-88d5-479c-bcb8-70ed07dc6e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:00.440366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44931be-69d6-4e8e-9a2c-1c4416fd27b6 service placement] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:00.466239 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1179/2348] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:28:00.521639 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15465968-88d5-479c-bcb8-70ed07dc6e58 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:00.526797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614afd37-407c-4494-8b02-a8caeed9d4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:00.531532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614afd37-407c-4494-8b02-a8caeed9d4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:00.531532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614afd37-407c-4494-8b02-a8caeed9d4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:00.531532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614afd37-407c-4494-8b02-a8caeed9d4c0 admin admin] PATH_INFO: `/v3/users/e81d8e2a42f34cc0b4f3ca54eb203f59` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:00.531532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614afd37-407c-4494-8b02-a8caeed9d4c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e81d8e2a42f34cc0b4f3ca54eb203f59)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:00.531532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614afd37-407c-4494-8b02-a8caeed9d4c0 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:00.536054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:00.720609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-614afd37-407c-4494-8b02-a8caeed9d4c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e81d8e2a42f34cc0b4f3ca54eb203f59'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:00.733751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-614afd37-407c-4494-8b02-a8caeed9d4c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e81d8e2a42f34cc0b4f3ca54eb203f59'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:00.740607 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-614afd37-407c-4494-8b02-a8caeed9d4c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e81d8e2a42f34cc0b4f3ca54eb203f59'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:00.838106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-614afd37-407c-4494-8b02-a8caeed9d4c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e81d8e2a42f34cc0b4f3ca54eb203f59'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:00.838802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-614afd37-407c-4494-8b02-a8caeed9d4c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e81d8e2a42f34cc0b4f3ca54eb203f59'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:00.846153 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1170/2349] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:28:00 2022] DELETE /identity/v3/users/e81d8e2a42f34cc0b4f3ca54eb203f59 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:28:01.114008 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44931be-69d6-4e8e-9a2c-1c4416fd27b6 service placement] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:01.124415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44931be-69d6-4e8e-9a2c-1c4416fd27b6 service placement] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:01.166415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:01.197866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78401e-0a56-4ea2-8888-0345c71e8896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:01.198515 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78401e-0a56-4ea2-8888-0345c71e8896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:01.198794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78401e-0a56-4ea2-8888-0345c71e8896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:01.199220 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78401e-0a56-4ea2-8888-0345c71e8896 admin admin] PATH_INFO: `/v3/projects/47fc574b240e4a09bb83d1d2f20558c4` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:01.200439 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78401e-0a56-4ea2-8888-0345c71e8896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47fc574b240e4a09bb83d1d2f20558c4)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:01.202456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78401e-0a56-4ea2-8888-0345c71e8896 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:01.364949 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cc78401e-0a56-4ea2-8888-0345c71e8896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47fc574b240e4a09bb83d1d2f20558c4', 'request_id': 'req-cc78401e-0a56-4ea2-8888-0345c71e8896'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:01.465660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cc78401e-0a56-4ea2-8888-0345c71e8896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47fc574b240e4a09bb83d1d2f20558c4', 'request_id': 'req-cc78401e-0a56-4ea2-8888-0345c71e8896'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:01.473085 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cc78401e-0a56-4ea2-8888-0345c71e8896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47fc574b240e4a09bb83d1d2f20558c4', 'request_id': 'req-cc78401e-0a56-4ea2-8888-0345c71e8896'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:01.486115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cc78401e-0a56-4ea2-8888-0345c71e8896 admin admin] The token cache is being invalidate because project 47fc574b240e4a09bb83d1d2f20558c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:28:01.486407 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cc78401e-0a56-4ea2-8888-0345c71e8896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:01.487073 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-cc78401e-0a56-4ea2-8888-0345c71e8896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:01.488694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1180/2350] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:28:01 2022] DELETE /identity/v3/projects/47fc574b240e4a09bb83d1d2f20558c4 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:28:01.495280 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0ddad-f2e6-4f8f-b26f-574fafe38a80 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:01.498923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0ddad-f2e6-4f8f-b26f-574fafe38a80 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:01.498923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0ddad-f2e6-4f8f-b26f-574fafe38a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:01.516121 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-55b0ddad-f2e6-4f8f-b26f-574fafe38a80 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:01.517442 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-55b0ddad-f2e6-4f8f-b26f-574fafe38a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:01.580573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1171/2351] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:01.588729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08319df9-d7c9-4e95-8d75-4d4625b889cb None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:01.589015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08319df9-d7c9-4e95-8d75-4d4625b889cb None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:01.589293 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08319df9-d7c9-4e95-8d75-4d4625b889cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:01.613610 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-08319df9-d7c9-4e95-8d75-4d4625b889cb None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:01.615601 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-08319df9-d7c9-4e95-8d75-4d4625b889cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:01.624098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1181/2352] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:28:01.628527 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0ddad-f2e6-4f8f-b26f-574fafe38a80 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:01.638612 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0ddad-f2e6-4f8f-b26f-574fafe38a80 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:01.643275 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64709348-83bb-45da-bf8a-25a6139d4994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:01.643874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64709348-83bb-45da-bf8a-25a6139d4994 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:01.644165 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64709348-83bb-45da-bf8a-25a6139d4994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:01.644433 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64709348-83bb-45da-bf8a-25a6139d4994 admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:01.645134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64709348-83bb-45da-bf8a-25a6139d4994 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:28:01.645440 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64709348-83bb-45da-bf8a-25a6139d4994 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:01.647052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64709348-83bb-45da-bf8a-25a6139d4994 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:01.659230 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1172/2353] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:28:01.663735 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08319df9-d7c9-4e95-8d75-4d4625b889cb None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:01.674255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08319df9-d7c9-4e95-8d75-4d4625b889cb None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:01.679420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199cb50-2cc1-4050-b8f0-3dfbb8ef7de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:01.681962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199cb50-2cc1-4050-b8f0-3dfbb8ef7de5 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:01.681962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199cb50-2cc1-4050-b8f0-3dfbb8ef7de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:01.681962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199cb50-2cc1-4050-b8f0-3dfbb8ef7de5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:01.681962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199cb50-2cc1-4050-b8f0-3dfbb8ef7de5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:01.683847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199cb50-2cc1-4050-b8f0-3dfbb8ef7de5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:01.830056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1182/2354] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:28:01 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:28:01.834205 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64709348-83bb-45da-bf8a-25a6139d4994 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:01.845084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64709348-83bb-45da-bf8a-25a6139d4994 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:01.905052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca3e7b-50ce-46dd-baad-d582444d99e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:01.905657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca3e7b-50ce-46dd-baad-d582444d99e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:01.905930 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca3e7b-50ce-46dd-baad-d582444d99e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:01.906205 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca3e7b-50ce-46dd-baad-d582444d99e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:01.906849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca3e7b-50ce-46dd-baad-d582444d99e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:01.908782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca3e7b-50ce-46dd-baad-d582444d99e3 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:02.037320 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:02.043243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1173/2355] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:28:01 2022] POST /identity/v3/users => generated 291 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:28:02.047406 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199cb50-2cc1-4050-b8f0-3dfbb8ef7de5 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:02.058495 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199cb50-2cc1-4050-b8f0-3dfbb8ef7de5 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:02.067863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702dc704-240a-485f-a211-56418b06f9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:02.067863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702dc704-240a-485f-a211-56418b06f9e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:02.067863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702dc704-240a-485f-a211-56418b06f9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:02.067863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702dc704-240a-485f-a211-56418b06f9e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:02.068579 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702dc704-240a-485f-a211-56418b06f9e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:02.068819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702dc704-240a-485f-a211-56418b06f9e9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:02.093774 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1183/2356] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:28:02.098068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca3e7b-50ce-46dd-baad-d582444d99e3 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:02.110307 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca3e7b-50ce-46dd-baad-d582444d99e3 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:02.120068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce51dbd-4c9a-4048-a14e-0ef5e154d5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:02.120711 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce51dbd-4c9a-4048-a14e-0ef5e154d5cc admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:02.121119 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce51dbd-4c9a-4048-a14e-0ef5e154d5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:02.121397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce51dbd-4c9a-4048-a14e-0ef5e154d5cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:02.122047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce51dbd-4c9a-4048-a14e-0ef5e154d5cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:02.124083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce51dbd-4c9a-4048-a14e-0ef5e154d5cc admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:02.147302 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1174/2357] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:28:02.153374 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702dc704-240a-485f-a211-56418b06f9e9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:02.165029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702dc704-240a-485f-a211-56418b06f9e9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:02.169799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aba979-97e8-4d1c-ae41-978514f8f90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:02.177451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aba979-97e8-4d1c-ae41-978514f8f90b admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:02.177451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aba979-97e8-4d1c-ae41-978514f8f90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:02.177451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aba979-97e8-4d1c-ae41-978514f8f90b admin admin] PATH_INFO: `/v3/projects/575748e2f3ee4df18ce5aa403175b66f/users/4682f63b56d24450b2605b5b4c2129d9/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:02.186306 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aba979-97e8-4d1c-ae41-978514f8f90b admin admin] RBAC: Authorizing `identity:create_grant(project_id=575748e2f3ee4df18ce5aa403175b66f, user_id=4682f63b56d24450b2605b5b4c2129d9, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:02.189686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aba979-97e8-4d1c-ae41-978514f8f90b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:02.284365 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1184/2358] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:28:02 2022] PUT /identity/v3/projects/575748e2f3ee4df18ce5aa403175b66f/users/4682f63b56d24450b2605b5b4c2129d9/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:28:02.289398 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9d71a-4ffe-4db7-b27c-0899e55ea5a9 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:02.289647 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9d71a-4ffe-4db7-b27c-0899e55ea5a9 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:02.289876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9d71a-4ffe-4db7-b27c-0899e55ea5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:02.309898 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-80a9d71a-4ffe-4db7-b27c-0899e55ea5a9 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:02.311138 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-80a9d71a-4ffe-4db7-b27c-0899e55ea5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:02.380325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1175/2359] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:02.390569 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aba979-97e8-4d1c-ae41-978514f8f90b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:02.392502 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e2aba979-97e8-4d1c-ae41-978514f8f90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:02.484054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aba979-97e8-4d1c-ae41-978514f8f90b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:02.488852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505b349-3449-4681-8714-50a1f70a97c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:02.489485 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505b349-3449-4681-8714-50a1f70a97c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:02.489811 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505b349-3449-4681-8714-50a1f70a97c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:02.490088 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505b349-3449-4681-8714-50a1f70a97c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:02.518934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505b349-3449-4681-8714-50a1f70a97c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:02.520473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505b349-3449-4681-8714-50a1f70a97c9 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:02.526090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:02.531698 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1185/2360] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:28:03.176188 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9d71a-4ffe-4db7-b27c-0899e55ea5a9 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:03.177722 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-80a9d71a-4ffe-4db7-b27c-0899e55ea5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:03.190492 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9d71a-4ffe-4db7-b27c-0899e55ea5a9 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:03.196126 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cea60-07b4-4edf-86d1-bb033692f1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:03.196574 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cea60-07b4-4edf-86d1-bb033692f1f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:03.196850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cea60-07b4-4edf-86d1-bb033692f1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:03.197124 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cea60-07b4-4edf-86d1-bb033692f1f6 admin admin] PATH_INFO: `/v3/users/54a6fee7082347e4957459e7ade47ece` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:03.198275 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cea60-07b4-4edf-86d1-bb033692f1f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54a6fee7082347e4957459e7ade47ece)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:03.200229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cea60-07b4-4edf-86d1-bb033692f1f6 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:03.521920 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-980cea60-07b4-4edf-86d1-bb033692f1f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54a6fee7082347e4957459e7ade47ece'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:03.552119 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-980cea60-07b4-4edf-86d1-bb033692f1f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54a6fee7082347e4957459e7ade47ece'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:03.562152 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-980cea60-07b4-4edf-86d1-bb033692f1f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54a6fee7082347e4957459e7ade47ece'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:03.649571 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-980cea60-07b4-4edf-86d1-bb033692f1f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54a6fee7082347e4957459e7ade47ece'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:03.650240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-980cea60-07b4-4edf-86d1-bb033692f1f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54a6fee7082347e4957459e7ade47ece'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:03.661526 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1176/2361] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:28:03 2022] DELETE /identity/v3/users/54a6fee7082347e4957459e7ade47ece => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:28:03.666291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b1e0f-428b-403a-bf84-7c5fac80d17e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:03.666933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b1e0f-428b-403a-bf84-7c5fac80d17e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:03.666933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b1e0f-428b-403a-bf84-7c5fac80d17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:03.690274 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8c8b1e0f-428b-403a-bf84-7c5fac80d17e None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:03.691661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8c8b1e0f-428b-403a-bf84-7c5fac80d17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:03.767245 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1186/2362] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:03.780402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cea60-07b4-4edf-86d1-bb033692f1f6 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:03.781861 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-980cea60-07b4-4edf-86d1-bb033692f1f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:03.852630 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cea60-07b4-4edf-86d1-bb033692f1f6 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:03.865229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89691eac-c0f2-46c0-b281-f3f11c05c1c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:03.865229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89691eac-c0f2-46c0-b281-f3f11c05c1c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:03.865229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89691eac-c0f2-46c0-b281-f3f11c05c1c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:03.865229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89691eac-c0f2-46c0-b281-f3f11c05c1c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:03.869779 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89691eac-c0f2-46c0-b281-f3f11c05c1c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:03.871261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89691eac-c0f2-46c0-b281-f3f11c05c1c0 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:03.883850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1177/2363] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:28:04.101635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5974c-0805-42d5-b2c2-51d3b8b1d93b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:04.109752 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5974c-0805-42d5-b2c2-51d3b8b1d93b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:04.109752 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5974c-0805-42d5-b2c2-51d3b8b1d93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:04.148963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-87b5974c-0805-42d5-b2c2-51d3b8b1d93b None None] MFA Rules not processed for user `4682f63b56d24450b2605b5b4c2129d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:04.161857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-87b5974c-0805-42d5-b2c2-51d3b8b1d93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:04.260231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1187/2364] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:28:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:04.273189 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89691eac-c0f2-46c0-b281-f3f11c05c1c0 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:04.275057 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-89691eac-c0f2-46c0-b281-f3f11c05c1c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:04.281330 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:04.347251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89691eac-c0f2-46c0-b281-f3f11c05c1c0 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:04.350868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83252b-8b9a-4dbc-9fd8-d2e40e3e004e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:04.352042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83252b-8b9a-4dbc-9fd8-d2e40e3e004e service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:04.352318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83252b-8b9a-4dbc-9fd8-d2e40e3e004e service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:04.352593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83252b-8b9a-4dbc-9fd8-d2e40e3e004e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:04.361666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83252b-8b9a-4dbc-9fd8-d2e40e3e004e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:04.363249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83252b-8b9a-4dbc-9fd8-d2e40e3e004e service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:04.373468 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1178/2365] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:28:04.446118 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5974c-0805-42d5-b2c2-51d3b8b1d93b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:04.448173 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-87b5974c-0805-42d5-b2c2-51d3b8b1d93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:04.465772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5974c-0805-42d5-b2c2-51d3b8b1d93b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:04.471158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c706d20-346b-402a-8ce6-401eefba136d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:04.471982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c706d20-346b-402a-8ce6-401eefba136d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:04.472338 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c706d20-346b-402a-8ce6-401eefba136d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:04.472680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c706d20-346b-402a-8ce6-401eefba136d admin admin] PATH_INFO: `/v3/projects/5facd1db989f4e21aedcfa7cfaca8eaa` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:04.473743 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c706d20-346b-402a-8ce6-401eefba136d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5facd1db989f4e21aedcfa7cfaca8eaa)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:04.475712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c706d20-346b-402a-8ce6-401eefba136d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:04.615362 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7c706d20-346b-402a-8ce6-401eefba136d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5facd1db989f4e21aedcfa7cfaca8eaa', 'request_id': 'req-7c706d20-346b-402a-8ce6-401eefba136d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:04.712637 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7c706d20-346b-402a-8ce6-401eefba136d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5facd1db989f4e21aedcfa7cfaca8eaa', 'request_id': 'req-7c706d20-346b-402a-8ce6-401eefba136d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:04.719958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7c706d20-346b-402a-8ce6-401eefba136d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5facd1db989f4e21aedcfa7cfaca8eaa', 'request_id': 'req-7c706d20-346b-402a-8ce6-401eefba136d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:04.732825 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7c706d20-346b-402a-8ce6-401eefba136d admin admin] The token cache is being invalidate because project 5facd1db989f4e21aedcfa7cfaca8eaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:28:04.733123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7c706d20-346b-402a-8ce6-401eefba136d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:04.733825 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7c706d20-346b-402a-8ce6-401eefba136d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:04.735636 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1188/2366] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:28:04 2022] DELETE /identity/v3/projects/5facd1db989f4e21aedcfa7cfaca8eaa => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:28:04.741899 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facc752-5ac3-469a-ab49-478ecf9f305a None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:04.742183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facc752-5ac3-469a-ab49-478ecf9f305a None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:04.744435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facc752-5ac3-469a-ab49-478ecf9f305a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:04.778448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-5facc752-5ac3-469a-ab49-478ecf9f305a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:04.790264 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5facc752-5ac3-469a-ab49-478ecf9f305a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:04.867257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1179/2367] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:04.868883 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ba95b-5b40-47d0-b575-6b6439397ee2 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:04.869161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ba95b-5b40-47d0-b575-6b6439397ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:04.869785 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ba95b-5b40-47d0-b575-6b6439397ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:04.903908 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f34ba95b-5b40-47d0-b575-6b6439397ee2 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:04.905174 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f34ba95b-5b40-47d0-b575-6b6439397ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:04.918079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1189/2368] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:28:04.922800 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5facc752-5ac3-469a-ab49-478ecf9f305a None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:04.929115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5facc752-5ac3-469a-ab49-478ecf9f305a None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:04.934238 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacc3d1-f18f-4ad5-a7ba-5c99f8e25ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:04.934836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacc3d1-f18f-4ad5-a7ba-5c99f8e25ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:04.935112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacc3d1-f18f-4ad5-a7ba-5c99f8e25ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:04.935413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacc3d1-f18f-4ad5-a7ba-5c99f8e25ce8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:04.936084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacc3d1-f18f-4ad5-a7ba-5c99f8e25ce8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:28:04.936441 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacc3d1-f18f-4ad5-a7ba-5c99f8e25ce8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:04.938216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacc3d1-f18f-4ad5-a7ba-5c99f8e25ce8 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:04.952823 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1180/2369] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:28:04.957381 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ba95b-5b40-47d0-b575-6b6439397ee2 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:04.968822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ba95b-5b40-47d0-b575-6b6439397ee2 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:04.973916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46d61a-4c1c-44d9-ba4b-4994e4612cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:04.974652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46d61a-4c1c-44d9-ba4b-4994e4612cff admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:04.974938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46d61a-4c1c-44d9-ba4b-4994e4612cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:04.975246 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46d61a-4c1c-44d9-ba4b-4994e4612cff admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:04.975896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46d61a-4c1c-44d9-ba4b-4994e4612cff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:04.977842 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46d61a-4c1c-44d9-ba4b-4994e4612cff admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:04.982694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:05.120442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1190/2370] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:28:04 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:28:05.124806 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacc3d1-f18f-4ad5-a7ba-5c99f8e25ce8 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:05.147365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacc3d1-f18f-4ad5-a7ba-5c99f8e25ce8 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:05.221646 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f566a-ae16-4e88-a00b-929e18bc4ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:05.222380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f566a-ae16-4e88-a00b-929e18bc4ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:05.222666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f566a-ae16-4e88-a00b-929e18bc4ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:05.222939 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f566a-ae16-4e88-a00b-929e18bc4ab1 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:05.223659 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f566a-ae16-4e88-a00b-929e18bc4ab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:05.225592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f566a-ae16-4e88-a00b-929e18bc4ab1 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:05.408195 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1181/2371] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:28:05 2022] POST /identity/v3/users => generated 287 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:28:05.412799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46d61a-4c1c-44d9-ba4b-4994e4612cff admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:05.424392 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46d61a-4c1c-44d9-ba4b-4994e4612cff admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:05.429224 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bdecf-f0b7-43a0-8bec-4eff9af41fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:05.429870 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bdecf-f0b7-43a0-8bec-4eff9af41fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:05.430663 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bdecf-f0b7-43a0-8bec-4eff9af41fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:05.430932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bdecf-f0b7-43a0-8bec-4eff9af41fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:05.431610 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2bdecf-f0b7-43a0-8bec-4eff9af41fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:05.433202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2bdecf-f0b7-43a0-8bec-4eff9af41fd8 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:05.444518 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1191/2372] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:28:05.448712 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f566a-ae16-4e88-a00b-929e18bc4ab1 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:05.459029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f566a-ae16-4e88-a00b-929e18bc4ab1 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:05.464553 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046a7c7-c775-41c4-89d8-e74cd28772b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:05.465152 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046a7c7-c775-41c4-89d8-e74cd28772b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:05.465433 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046a7c7-c775-41c4-89d8-e74cd28772b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:05.465707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046a7c7-c775-41c4-89d8-e74cd28772b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:05.466345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5046a7c7-c775-41c4-89d8-e74cd28772b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:05.468136 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5046a7c7-c775-41c4-89d8-e74cd28772b5 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:05.480185 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1182/2373] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:28:05.484427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bdecf-f0b7-43a0-8bec-4eff9af41fd8 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:05.494338 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bdecf-f0b7-43a0-8bec-4eff9af41fd8 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:05.499016 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc814b8-96ce-49eb-9adc-f6d09fab75fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:05.499704 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc814b8-96ce-49eb-9adc-f6d09fab75fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:05.499987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc814b8-96ce-49eb-9adc-f6d09fab75fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:05.500260 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc814b8-96ce-49eb-9adc-f6d09fab75fa admin admin] PATH_INFO: `/v3/projects/0874e13ae9984c9fbcedb57f7717a73c/users/4ef532361ce8429ea43232ee36d1a3d4/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:05.515046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc814b8-96ce-49eb-9adc-f6d09fab75fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0874e13ae9984c9fbcedb57f7717a73c, user_id=4ef532361ce8429ea43232ee36d1a3d4, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:05.519497 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc814b8-96ce-49eb-9adc-f6d09fab75fa admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:05.630278 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1192/2374] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:28:05 2022] PUT /identity/v3/projects/0874e13ae9984c9fbcedb57f7717a73c/users/4ef532361ce8429ea43232ee36d1a3d4/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:28:05.641043 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0b80-34b9-4b6e-b814-71ba0c996859 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:05.641607 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0b80-34b9-4b6e-b814-71ba0c996859 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:05.641952 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0b80-34b9-4b6e-b814-71ba0c996859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:05.669575 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:05.672923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-1f5b0b80-34b9-4b6e-b814-71ba0c996859 None None] MFA Rules not processed for user `4ef532361ce8429ea43232ee36d1a3d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:05.674369 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1f5b0b80-34b9-4b6e-b814-71ba0c996859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:05.746063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1183/2375] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:28:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:05.759775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc814b8-96ce-49eb-9adc-f6d09fab75fa admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:05.763291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-afc814b8-96ce-49eb-9adc-f6d09fab75fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:05.824288 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc814b8-96ce-49eb-9adc-f6d09fab75fa admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:05.828277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562522d7-e7b4-45f2-b404-8a374e0c1149 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:05.828897 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562522d7-e7b4-45f2-b404-8a374e0c1149 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:05.829184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562522d7-e7b4-45f2-b404-8a374e0c1149 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:05.829462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562522d7-e7b4-45f2-b404-8a374e0c1149 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:05.838844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562522d7-e7b4-45f2-b404-8a374e0c1149 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:05.840496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562522d7-e7b4-45f2-b404-8a374e0c1149 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:05.849345 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1193/2376] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:28:06.182433 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f830b-93c4-4bdb-bf57-de52de5dc44c None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:06.182668 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f830b-93c4-4bdb-bf57-de52de5dc44c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:06.182952 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f830b-93c4-4bdb-bf57-de52de5dc44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:06.205150 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-f34f830b-93c4-4bdb-bf57-de52de5dc44c None None] MFA Rules not processed for user `4682f63b56d24450b2605b5b4c2129d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:06.206583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f34f830b-93c4-4bdb-bf57-de52de5dc44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:06.288284 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1184/2377] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:28:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:07.678376 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71cde4-b117-4bd5-97b6-412c2f492306 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:07.685450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71cde4-b117-4bd5-97b6-412c2f492306 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:07.685450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71cde4-b117-4bd5-97b6-412c2f492306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:07.698901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-9f71cde4-b117-4bd5-97b6-412c2f492306 None None] MFA Rules not processed for user `4ef532361ce8429ea43232ee36d1a3d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:07.700170 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9f71cde4-b117-4bd5-97b6-412c2f492306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:07.707074 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1194/2378] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:28:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:28:16.275429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=51 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:17.693285 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=50,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:19.663437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f830b-93c4-4bdb-bf57-de52de5dc44c None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:19.663437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f34f830b-93c4-4bdb-bf57-de52de5dc44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:19.735701 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f830b-93c4-4bdb-bf57-de52de5dc44c None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:19.739580 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897000a-686d-4210-9183-5fb3da8241c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:19.740196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897000a-686d-4210-9183-5fb3da8241c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:19.740472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897000a-686d-4210-9183-5fb3da8241c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:19.740746 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897000a-686d-4210-9183-5fb3da8241c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:19.742562 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-6897000a-686d-4210-9183-5fb3da8241c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:19.819614 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897000a-686d-4210-9183-5fb3da8241c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:19.822510 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897000a-686d-4210-9183-5fb3da8241c4 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:19.832598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1185/2379] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:28:29.676841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775efcc-e69a-4201-86e9-1945dd5389ec None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:29.677355 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775efcc-e69a-4201-86e9-1945dd5389ec None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:29.677471 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775efcc-e69a-4201-86e9-1945dd5389ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:29.698130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-5775efcc-e69a-4201-86e9-1945dd5389ec None None] MFA Rules not processed for user `5a667fa07b1148998959835d9995ad60`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:29.700410 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5775efcc-e69a-4201-86e9-1945dd5389ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:29.758845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1195/2380] 10.222.0.78 () {58 vars in 1009 bytes} [Mon Sep 19 11:28:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:28:29.780470 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897000a-686d-4210-9183-5fb3da8241c4 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:29.782767 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-6897000a-686d-4210-9183-5fb3da8241c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:29.857565 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897000a-686d-4210-9183-5fb3da8241c4 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:29.862745 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab883f9c-8ea8-41b9-a41e-1df2d163b11a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf462b2aa3fc4e2ca58851549cf0c644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:29.863392 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab883f9c-8ea8-41b9-a41e-1df2d163b11a service swift] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:29.863670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab883f9c-8ea8-41b9-a41e-1df2d163b11a service swift] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:29.863946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab883f9c-8ea8-41b9-a41e-1df2d163b11a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:29.873244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab883f9c-8ea8-41b9-a41e-1df2d163b11a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:29.874837 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab883f9c-8ea8-41b9-a41e-1df2d163b11a service swift] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:29.885179 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1186/2381] 10.222.0.78 () {60 vars in 1384 bytes} [Mon Sep 19 11:28:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:28:30.922322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433a974-ae53-47e3-8148-9804aeb1df95 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:30.927347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433a974-ae53-47e3-8148-9804aeb1df95 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:30.927347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433a974-ae53-47e3-8148-9804aeb1df95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:30.943977 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c433a974-ae53-47e3-8148-9804aeb1df95 None None] MFA Rules not processed for user `5a667fa07b1148998959835d9995ad60`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:30.947394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c433a974-ae53-47e3-8148-9804aeb1df95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:30.957945 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1196/2382] 10.222.0.78 () {58 vars in 967 bytes} [Mon Sep 19 11:28:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:30.978854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab883f9c-8ea8-41b9-a41e-1df2d163b11a service swift] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:30.993580 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab883f9c-8ea8-41b9-a41e-1df2d163b11a service swift] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:31.000400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd775295-71ac-445b-8087-9646c124c9fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf462b2aa3fc4e2ca58851549cf0c644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:31.001090 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd775295-71ac-445b-8087-9646c124c9fd service swift] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:31.001438 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd775295-71ac-445b-8087-9646c124c9fd service swift] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:31.001762 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd775295-71ac-445b-8087-9646c124c9fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:31.016447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd775295-71ac-445b-8087-9646c124c9fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:31.017971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd775295-71ac-445b-8087-9646c124c9fd service swift] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:31.027150 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1187/2383] 10.222.0.78 () {60 vars in 1384 bytes} [Mon Sep 19 11:28:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 11:28:31.751047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f999e9-5be3-4392-b65d-c9b140dcd9bc None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:31.751406 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f999e9-5be3-4392-b65d-c9b140dcd9bc None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:31.751648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f999e9-5be3-4392-b65d-c9b140dcd9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:31.771923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-45f999e9-5be3-4392-b65d-c9b140dcd9bc None None] MFA Rules not processed for user `5a667fa07b1148998959835d9995ad60`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:31.773178 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-45f999e9-5be3-4392-b65d-c9b140dcd9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:31.781798 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1197/2384] 10.222.0.78 () {58 vars in 1009 bytes} [Mon Sep 19 11:28:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:31.793422 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd775295-71ac-445b-8087-9646c124c9fd service swift] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:31.804654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd775295-71ac-445b-8087-9646c124c9fd service swift] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:31.809854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989123c-ba98-4206-988d-e32f7057cde9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf462b2aa3fc4e2ca58851549cf0c644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:31.810450 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989123c-ba98-4206-988d-e32f7057cde9 service swift] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:31.810736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989123c-ba98-4206-988d-e32f7057cde9 service swift] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:31.811008 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989123c-ba98-4206-988d-e32f7057cde9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:31.820086 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989123c-ba98-4206-988d-e32f7057cde9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:31.821716 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989123c-ba98-4206-988d-e32f7057cde9 service swift] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:31.830822 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1188/2385] 10.222.0.78 () {60 vars in 1384 bytes} [Mon Sep 19 11:28:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:28:32.461809 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3f018-ab33-4ee4-9bc3-e80d2d852983 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:32.462289 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3f018-ab33-4ee4-9bc3-e80d2d852983 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:32.462412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3f018-ab33-4ee4-9bc3-e80d2d852983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:32.484674 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8ca3f018-ab33-4ee4-9bc3-e80d2d852983 None None] MFA Rules not processed for user `5a667fa07b1148998959835d9995ad60`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:32.485932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8ca3f018-ab33-4ee4-9bc3-e80d2d852983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:32.494022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1198/2386] 10.222.0.78 () {58 vars in 967 bytes} [Mon Sep 19 11:28:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:32.507525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989123c-ba98-4206-988d-e32f7057cde9 service swift] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:32.518237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989123c-ba98-4206-988d-e32f7057cde9 service swift] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:32.523480 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e78f5-dd3b-45d3-89c1-ed04d364da4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf462b2aa3fc4e2ca58851549cf0c644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:32.524097 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e78f5-dd3b-45d3-89c1-ed04d364da4f service swift] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:32.524377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e78f5-dd3b-45d3-89c1-ed04d364da4f service swift] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:32.524649 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e78f5-dd3b-45d3-89c1-ed04d364da4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:32.533267 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e78f5-dd3b-45d3-89c1-ed04d364da4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:32.534886 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e78f5-dd3b-45d3-89c1-ed04d364da4f service swift] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:32.539800 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:32.543768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1189/2387] 10.222.0.78 () {60 vars in 1384 bytes} [Mon Sep 19 11:28:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 11:28:35.452573 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3f018-ab33-4ee4-9bc3-e80d2d852983 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:35.523927 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3f018-ab33-4ee4-9bc3-e80d2d852983 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:35.529000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299bc62-559e-4451-9a62-661f4a2f2287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:35.529636 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299bc62-559e-4451-9a62-661f4a2f2287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:35.529976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299bc62-559e-4451-9a62-661f4a2f2287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:35.530257 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299bc62-559e-4451-9a62-661f4a2f2287 admin admin] PATH_INFO: `/v3/users/4ef532361ce8429ea43232ee36d1a3d4` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:35.531508 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0299bc62-559e-4451-9a62-661f4a2f2287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ef532361ce8429ea43232ee36d1a3d4)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:35.533706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0299bc62-559e-4451-9a62-661f4a2f2287 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:35.721863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0299bc62-559e-4451-9a62-661f4a2f2287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ef532361ce8429ea43232ee36d1a3d4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:35.737257 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0299bc62-559e-4451-9a62-661f4a2f2287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ef532361ce8429ea43232ee36d1a3d4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:35.744657 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0299bc62-559e-4451-9a62-661f4a2f2287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ef532361ce8429ea43232ee36d1a3d4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:35.814683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0299bc62-559e-4451-9a62-661f4a2f2287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ef532361ce8429ea43232ee36d1a3d4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:35.815381 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0299bc62-559e-4451-9a62-661f4a2f2287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ef532361ce8429ea43232ee36d1a3d4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:35.822196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1199/2388] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:28:35 2022] DELETE /identity/v3/users/4ef532361ce8429ea43232ee36d1a3d4 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:28:35.827647 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd46f3-c329-4b0d-9919-c52d12249c7d None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:35.828146 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd46f3-c329-4b0d-9919-c52d12249c7d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:35.828449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd46f3-c329-4b0d-9919-c52d12249c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:35.848898 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-fdbd46f3-c329-4b0d-9919-c52d12249c7d None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:35.855670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-fdbd46f3-c329-4b0d-9919-c52d12249c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:35.914010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1190/2389] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:35.923778 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299bc62-559e-4451-9a62-661f4a2f2287 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:35.925212 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0299bc62-559e-4451-9a62-661f4a2f2287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:35.935772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:35.992618 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299bc62-559e-4451-9a62-661f4a2f2287 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:35.996654 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a53e7-2adf-42d3-8eb1-1e6f83748d2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:35.997838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a53e7-2adf-42d3-8eb1-1e6f83748d2d service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:35.998120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a53e7-2adf-42d3-8eb1-1e6f83748d2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:35.998399 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a53e7-2adf-42d3-8eb1-1e6f83748d2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:36.011310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278a53e7-2adf-42d3-8eb1-1e6f83748d2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:36.012907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278a53e7-2adf-42d3-8eb1-1e6f83748d2d service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:36.023322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1200/2390] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:28:36.300650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd46f3-c329-4b0d-9919-c52d12249c7d None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:36.301820 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-fdbd46f3-c329-4b0d-9919-c52d12249c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:36.315077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd46f3-c329-4b0d-9919-c52d12249c7d None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:36.320220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367a06a-5ae4-403f-8319-e2e4068a5922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:36.320799 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367a06a-5ae4-403f-8319-e2e4068a5922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:36.321096 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367a06a-5ae4-403f-8319-e2e4068a5922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:36.321784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367a06a-5ae4-403f-8319-e2e4068a5922 admin admin] PATH_INFO: `/v3/projects/0874e13ae9984c9fbcedb57f7717a73c` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:36.322837 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367a06a-5ae4-403f-8319-e2e4068a5922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0874e13ae9984c9fbcedb57f7717a73c)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:36.324907 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367a06a-5ae4-403f-8319-e2e4068a5922 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:36.441759 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-3367a06a-5ae4-403f-8319-e2e4068a5922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0874e13ae9984c9fbcedb57f7717a73c', 'request_id': 'req-3367a06a-5ae4-403f-8319-e2e4068a5922'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:36.448928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-3367a06a-5ae4-403f-8319-e2e4068a5922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0874e13ae9984c9fbcedb57f7717a73c', 'request_id': 'req-3367a06a-5ae4-403f-8319-e2e4068a5922'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:36.527924 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-3367a06a-5ae4-403f-8319-e2e4068a5922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0874e13ae9984c9fbcedb57f7717a73c', 'request_id': 'req-3367a06a-5ae4-403f-8319-e2e4068a5922'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:36.545702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-3367a06a-5ae4-403f-8319-e2e4068a5922 admin admin] The token cache is being invalidate because project 0874e13ae9984c9fbcedb57f7717a73c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:28:36.545995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-3367a06a-5ae4-403f-8319-e2e4068a5922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:36.546676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-3367a06a-5ae4-403f-8319-e2e4068a5922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:36.548209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1191/2391] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:28:36 2022] DELETE /identity/v3/projects/0874e13ae9984c9fbcedb57f7717a73c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:28:39.849149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a53e7-2adf-42d3-8eb1-1e6f83748d2d service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:39.865247 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a53e7-2adf-42d3-8eb1-1e6f83748d2d service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:39.917617 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b05cc7-db52-482a-b6bf-b6888ea0b227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:39.918225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b05cc7-db52-482a-b6bf-b6888ea0b227 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:39.918503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b05cc7-db52-482a-b6bf-b6888ea0b227 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:39.918773 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b05cc7-db52-482a-b6bf-b6888ea0b227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:39.921201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-69b05cc7-db52-482a-b6bf-b6888ea0b227 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:39.983114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b05cc7-db52-482a-b6bf-b6888ea0b227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:39.985005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b05cc7-db52-482a-b6bf-b6888ea0b227 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:39.994432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1201/2392] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:28:46.541644 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:49.990262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=74 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:09.430207 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9a848-702b-48bb-8b13-a86c77254c05 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:09.430776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9a848-702b-48bb-8b13-a86c77254c05 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:09.430776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9a848-702b-48bb-8b13-a86c77254c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:09.451576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-20f9a848-702b-48bb-8b13-a86c77254c05 None None] MFA Rules not processed for user `5a667fa07b1148998959835d9995ad60`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:09.453379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-20f9a848-702b-48bb-8b13-a86c77254c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:09.515933 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1192/2393] 10.222.0.78 () {58 vars in 1009 bytes} [Mon Sep 19 11:29:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:09.529984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b05cc7-db52-482a-b6bf-b6888ea0b227 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:09.531503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-69b05cc7-db52-482a-b6bf-b6888ea0b227 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:09.611639 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b05cc7-db52-482a-b6bf-b6888ea0b227 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:09.616936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755884f8-cfdf-458b-bc9c-7205dfdec4ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf462b2aa3fc4e2ca58851549cf0c644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:09.617713 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755884f8-cfdf-458b-bc9c-7205dfdec4ea service swift] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:09.617998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755884f8-cfdf-458b-bc9c-7205dfdec4ea service swift] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:09.618279 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755884f8-cfdf-458b-bc9c-7205dfdec4ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:09.628531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755884f8-cfdf-458b-bc9c-7205dfdec4ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:09.630131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755884f8-cfdf-458b-bc9c-7205dfdec4ea service swift] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:09.638699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1202/2394] 10.222.0.78 () {60 vars in 1384 bytes} [Mon Sep 19 11:29:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:29:14.380757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed83984-6d1e-4ec1-83bc-367d51db39e7 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:14.381382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed83984-6d1e-4ec1-83bc-367d51db39e7 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:14.381526 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed83984-6d1e-4ec1-83bc-367d51db39e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:14.413056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-bed83984-6d1e-4ec1-83bc-367d51db39e7 None None] MFA Rules not processed for user `d6fee905418f4d879da460b6758d4f23`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:14.415612 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-bed83984-6d1e-4ec1-83bc-367d51db39e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:14.503855 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1193/2395] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:29:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:19.635714 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=32 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:24.493550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=50 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:29.666706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5ba24-28f6-4584-8018-557423b604d4 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:29.667304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5ba24-28f6-4584-8018-557423b604d4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:29.667479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5ba24-28f6-4584-8018-557423b604d4 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:29.668915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1203/2396] 10.222.0.78 () {54 vars in 921 bytes} [Mon Sep 19 11:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:29:29.677319 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e496e-4773-4e3a-a147-a1501a29bc9f None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:29.678901 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e496e-4773-4e3a-a147-a1501a29bc9f None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:29.679160 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e496e-4773-4e3a-a147-a1501a29bc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:29.700327 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-682e496e-4773-4e3a-a147-a1501a29bc9f None None] MFA Rules not processed for user `aba1968529574fe5b4c5d114c874f428`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:29.701809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-682e496e-4773-4e3a-a147-a1501a29bc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:29.770073 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1194/2397] 10.222.0.78 () {58 vars in 1018 bytes} [Mon Sep 19 11:29:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:29.781887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5ba24-28f6-4584-8018-557423b604d4 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:29.788296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d7d5ba24-28f6-4584-8018-557423b604d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:29.843158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5ba24-28f6-4584-8018-557423b604d4 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:29.848076 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc0120-3845-4366-8799-80b3e7b6e436 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558e3b6dd9754c57ad028aa431a68b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:29.848683 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc0120-3845-4366-8799-80b3e7b6e436 service glance] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:29.848959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc0120-3845-4366-8799-80b3e7b6e436 service glance] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:29.849229 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc0120-3845-4366-8799-80b3e7b6e436 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:29.859752 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc0120-3845-4366-8799-80b3e7b6e436 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:29.861208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc0120-3845-4366-8799-80b3e7b6e436 service glance] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:29.872567 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1204/2398] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:29:30.074142 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd929bb-7444-4fc8-a759-542477cfc145 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:30.074408 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd929bb-7444-4fc8-a759-542477cfc145 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:30.074689 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd929bb-7444-4fc8-a759-542477cfc145 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:30.076130 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1195/2399] 10.222.0.78 () {54 vars in 921 bytes} [Mon Sep 19 11:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:29:30.081706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16820e5-962d-488a-8819-90ed80ee2d76 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:30.081990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16820e5-962d-488a-8819-90ed80ee2d76 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:30.082263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16820e5-962d-488a-8819-90ed80ee2d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:30.102607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b16820e5-962d-488a-8819-90ed80ee2d76 None None] MFA Rules not processed for user `dbf0049a3e3a442e945f6207477a7fd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:30.103890 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b16820e5-962d-488a-8819-90ed80ee2d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:30.110655 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1205/2400] 10.222.0.78 () {58 vars in 1018 bytes} [Mon Sep 19 11:29:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:29:30.126021 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd929bb-7444-4fc8-a759-542477cfc145 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:30.127783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-3fd929bb-7444-4fc8-a759-542477cfc145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:30.141020 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd929bb-7444-4fc8-a759-542477cfc145 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:30.144787 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf989d-6aab-4520-abf4-5b7ea41e8451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:30.145390 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf989d-6aab-4520-abf4-5b7ea41e8451 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:30.145671 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf989d-6aab-4520-abf4-5b7ea41e8451 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:30.147985 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf989d-6aab-4520-abf4-5b7ea41e8451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:30.153279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf989d-6aab-4520-abf4-5b7ea41e8451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:30.154756 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf989d-6aab-4520-abf4-5b7ea41e8451 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:30.163305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1196/2401] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:29:30.248682 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb2d88-1b6f-41b3-8978-822fc6620537 None None] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:30.248970 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb2d88-1b6f-41b3-8978-822fc6620537 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:30.249247 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb2d88-1b6f-41b3-8978-822fc6620537 None None] PATH_INFO: `/` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:30.250759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1206/2402] 10.222.0.78 () {54 vars in 921 bytes} [Mon Sep 19 11:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:29:30.255826 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544d544-6b3d-4448-8f4d-8f0cb713a9ba None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:30.256135 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544d544-6b3d-4448-8f4d-8f0cb713a9ba None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:30.256410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544d544-6b3d-4448-8f4d-8f0cb713a9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:30.277070 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-b544d544-6b3d-4448-8f4d-8f0cb713a9ba None None] MFA Rules not processed for user `aba1968529574fe5b4c5d114c874f428`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:30.278289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b544d544-6b3d-4448-8f4d-8f0cb713a9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:30.288353 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1197/2403] 10.222.0.78 () {58 vars in 1018 bytes} [Mon Sep 19 11:29:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:30.298857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb2d88-1b6f-41b3-8978-822fc6620537 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:30.310014 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb2d88-1b6f-41b3-8978-822fc6620537 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:30.313633 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101db57-59ab-4d75-8c9c-49daea66acd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:30.314274 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101db57-59ab-4d75-8c9c-49daea66acd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:30.319885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101db57-59ab-4d75-8c9c-49daea66acd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:30.319885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101db57-59ab-4d75-8c9c-49daea66acd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:30.325575 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101db57-59ab-4d75-8c9c-49daea66acd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:30.327000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101db57-59ab-4d75-8c9c-49daea66acd0 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:30.338866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1207/2404] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:29:34.149112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27bf0c-e843-4cc1-9eb6-047250be08bd None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:34.149723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27bf0c-e843-4cc1-9eb6-047250be08bd None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:34.149723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27bf0c-e843-4cc1-9eb6-047250be08bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:34.170042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-ac27bf0c-e843-4cc1-9eb6-047250be08bd None None] MFA Rules not processed for user `4682f63b56d24450b2605b5b4c2129d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:34.171282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ac27bf0c-e843-4cc1-9eb6-047250be08bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:34.228320 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1198/2405] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:29:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:36.146836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c99f69-0edd-4586-b23e-a1eaaacdaae9 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:36.147363 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c99f69-0edd-4586-b23e-a1eaaacdaae9 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:36.147475 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c99f69-0edd-4586-b23e-a1eaaacdaae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:36.166906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-25c99f69-0edd-4586-b23e-a1eaaacdaae9 None None] MFA Rules not processed for user `5a667fa07b1148998959835d9995ad60`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:36.168135 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-25c99f69-0edd-4586-b23e-a1eaaacdaae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:36.175810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1208/2406] 10.222.0.78 () {58 vars in 1009 bytes} [Mon Sep 19 11:29:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:36.197514 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27bf0c-e843-4cc1-9eb6-047250be08bd None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:36.209867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27bf0c-e843-4cc1-9eb6-047250be08bd None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:36.215207 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd588821-9350-448e-a91c-1d3527fa9195 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf462b2aa3fc4e2ca58851549cf0c644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:36.215674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd588821-9350-448e-a91c-1d3527fa9195 service swift] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:36.215956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd588821-9350-448e-a91c-1d3527fa9195 service swift] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:36.216229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd588821-9350-448e-a91c-1d3527fa9195 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:36.225328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd588821-9350-448e-a91c-1d3527fa9195 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:36.228635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd588821-9350-448e-a91c-1d3527fa9195 service swift] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:36.235231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1199/2407] 10.222.0.78 () {60 vars in 1384 bytes} [Mon Sep 19 11:29:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:29:46.162123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=38 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:46.232249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=71 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:54.650599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c99f69-0edd-4586-b23e-a1eaaacdaae9 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:54.660521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c99f69-0edd-4586-b23e-a1eaaacdaae9 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:54.664269 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfe84a-448c-4076-be98-1655d5a43fa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:54.672179 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfe84a-448c-4076-be98-1655d5a43fa9 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:54.672179 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfe84a-448c-4076-be98-1655d5a43fa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:54.672179 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfe84a-448c-4076-be98-1655d5a43fa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:54.672179 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-68bfe84a-448c-4076-be98-1655d5a43fa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:54.684320 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfe84a-448c-4076-be98-1655d5a43fa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:54.685757 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfe84a-448c-4076-be98-1655d5a43fa9 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:54.697583 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1209/2408] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:29:56.474588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c50c8-58ec-47eb-bde6-a55ed137dd97 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:56.475191 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c50c8-58ec-47eb-bde6-a55ed137dd97 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:56.475191 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c50c8-58ec-47eb-bde6-a55ed137dd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:56.494868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-426c50c8-58ec-47eb-bde6-a55ed137dd97 None None] MFA Rules not processed for user `405b3189506a44bc93555c40917b4cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:56.496134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-426c50c8-58ec-47eb-bde6-a55ed137dd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:56.554708 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1200/2409] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:29:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:56.576478 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfe84a-448c-4076-be98-1655d5a43fa9 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:56.585998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfe84a-448c-4076-be98-1655d5a43fa9 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:56.589697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87020e80-8ec3-4d90-86ff-19545750481d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:56.590608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87020e80-8ec3-4d90-86ff-19545750481d service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:56.590608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87020e80-8ec3-4d90-86ff-19545750481d service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:56.590779 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87020e80-8ec3-4d90-86ff-19545750481d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:56.592671 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-87020e80-8ec3-4d90-86ff-19545750481d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:56.607541 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87020e80-8ec3-4d90-86ff-19545750481d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:56.608955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87020e80-8ec3-4d90-86ff-19545750481d service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:56.624267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1210/2410] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:02.238437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c50c8-58ec-47eb-bde6-a55ed137dd97 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:02.239878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-426c50c8-58ec-47eb-bde6-a55ed137dd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:02.297150 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c50c8-58ec-47eb-bde6-a55ed137dd97 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:02.301050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0e9fc-97de-41c0-860c-3241070bd0dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:02.301606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0e9fc-97de-41c0-860c-3241070bd0dd service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:02.301882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0e9fc-97de-41c0-860c-3241070bd0dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:02.302158 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0e9fc-97de-41c0-860c-3241070bd0dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:02.303923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-0cf0e9fc-97de-41c0-860c-3241070bd0dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:02.315632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0e9fc-97de-41c0-860c-3241070bd0dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:02.317071 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0e9fc-97de-41c0-860c-3241070bd0dd service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:02.326740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1201/2411] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:02.985344 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc85c7a-d96f-4fa2-aa0d-70557b2f0fcc None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:02.986231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc85c7a-d96f-4fa2-aa0d-70557b2f0fcc None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:02.986587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc85c7a-d96f-4fa2-aa0d-70557b2f0fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:03.008657 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-cdc85c7a-d96f-4fa2-aa0d-70557b2f0fcc None None] MFA Rules not processed for user `5a667fa07b1148998959835d9995ad60`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:03.009896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-cdc85c7a-d96f-4fa2-aa0d-70557b2f0fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:03.021664 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1211/2412] 10.222.0.78 () {58 vars in 967 bytes} [Mon Sep 19 11:30:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:03.034206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0e9fc-97de-41c0-860c-3241070bd0dd service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:03.046416 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0e9fc-97de-41c0-860c-3241070bd0dd service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:03.052501 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06e0da-f74d-4fc0-af24-36f92801c72e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf462b2aa3fc4e2ca58851549cf0c644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:03.053117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06e0da-f74d-4fc0-af24-36f92801c72e service swift] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:03.053409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06e0da-f74d-4fc0-af24-36f92801c72e service swift] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:03.053691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06e0da-f74d-4fc0-af24-36f92801c72e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:03.062373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06e0da-f74d-4fc0-af24-36f92801c72e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:03.064006 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06e0da-f74d-4fc0-af24-36f92801c72e service swift] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:03.082200 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1202/2413] 10.222.0.78 () {60 vars in 1384 bytes} [Mon Sep 19 11:30:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:30:08.907635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc85c7a-d96f-4fa2-aa0d-70557b2f0fcc None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:08.909152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-cdc85c7a-d96f-4fa2-aa0d-70557b2f0fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:08.924953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc85c7a-d96f-4fa2-aa0d-70557b2f0fcc None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:08.930969 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f7b6f-3bf2-4726-85de-7ea5fbe70827 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:08.932718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f7b6f-3bf2-4726-85de-7ea5fbe70827 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:08.932718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f7b6f-3bf2-4726-85de-7ea5fbe70827 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:08.932718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f7b6f-3bf2-4726-85de-7ea5fbe70827 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:08.934020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-807f7b6f-3bf2-4726-85de-7ea5fbe70827 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:08.945392 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f7b6f-3bf2-4726-85de-7ea5fbe70827 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:08.946980 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f7b6f-3bf2-4726-85de-7ea5fbe70827 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:08.956950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1212/2414] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:13.073753 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=57 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:14.789712 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06e0da-f74d-4fc0-af24-36f92801c72e service swift] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:14.799064 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06e0da-f74d-4fc0-af24-36f92801c72e service swift] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:14.802927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5b32a-b208-4e38-a1bc-259a0aaaf4fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:14.803575 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5b32a-b208-4e38-a1bc-259a0aaaf4fd service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:14.803860 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5b32a-b208-4e38-a1bc-259a0aaaf4fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:14.804135 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5b32a-b208-4e38-a1bc-259a0aaaf4fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:14.805883 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-57c5b32a-b208-4e38-a1bc-259a0aaaf4fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:14.815675 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5b32a-b208-4e38-a1bc-259a0aaaf4fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:14.817645 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5b32a-b208-4e38-a1bc-259a0aaaf4fd service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:14.825801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1203/2415] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:18.952260 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=21 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:24.822128 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=6 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:31:15.585496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f7b6f-3bf2-4726-85de-7ea5fbe70827 service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:15.597343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f7b6f-3bf2-4726-85de-7ea5fbe70827 service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:15.602802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634fe21-e167-40c7-9438-e882c71ab435 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:15.604626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634fe21-e167-40c7-9438-e882c71ab435 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:15.604626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634fe21-e167-40c7-9438-e882c71ab435 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:15.604626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634fe21-e167-40c7-9438-e882c71ab435 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:15.606146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c634fe21-e167-40c7-9438-e882c71ab435 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:15.618933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634fe21-e167-40c7-9438-e882c71ab435 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:15.620479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634fe21-e167-40c7-9438-e882c71ab435 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:15.630929 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1213/2416] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:31:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:31:25.626621 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=6 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:32:03.719672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5b32a-b208-4e38-a1bc-259a0aaaf4fd service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:32:03.721172 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-57c5b32a-b208-4e38-a1bc-259a0aaaf4fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:32:03.737279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5b32a-b208-4e38-a1bc-259a0aaaf4fd service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:32:03.743455 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9a712-4938-4362-a8e4-70a218a2f56b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:32:03.744220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9a712-4938-4362-a8e4-70a218a2f56b service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:32:03.744341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9a712-4938-4362-a8e4-70a218a2f56b service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:32:03.744590 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9a712-4938-4362-a8e4-70a218a2f56b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:32:03.746271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-3dc9a712-4938-4362-a8e4-70a218a2f56b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:32:03.804339 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9a712-4938-4362-a8e4-70a218a2f56b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:32:03.805418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9a712-4938-4362-a8e4-70a218a2f56b service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:32:03.814445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1204/2417] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:32:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:32:07.126244 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634fe21-e167-40c7-9438-e882c71ab435 service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:32:07.140195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634fe21-e167-40c7-9438-e882c71ab435 service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:32:07.145738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ec09a-ad89-4d49-a5a2-434c32e1014b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:32:07.146343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ec09a-ad89-4d49-a5a2-434c32e1014b service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:32:07.146623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ec09a-ad89-4d49-a5a2-434c32e1014b service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:32:07.146895 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ec09a-ad89-4d49-a5a2-434c32e1014b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:32:07.148715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e43ec09a-ad89-4d49-a5a2-434c32e1014b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:32:07.213193 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ec09a-ad89-4d49-a5a2-434c32e1014b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:32:07.214786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ec09a-ad89-4d49-a5a2-434c32e1014b service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:32:07.224302 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1214/2418] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:32:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:32:13.810208 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=28 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:32:17.219735 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=28 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:32:38.452005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9a712-4938-4362-a8e4-70a218a2f56b service nova] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:32:38.464980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9a712-4938-4362-a8e4-70a218a2f56b service nova] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:32:38.470978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd1c33-d45f-4330-9ad0-71d2cbe8d65d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:32:38.471624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd1c33-d45f-4330-9ad0-71d2cbe8d65d service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:32:38.471903 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd1c33-d45f-4330-9ad0-71d2cbe8d65d service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:32:38.472167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd1c33-d45f-4330-9ad0-71d2cbe8d65d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:32:38.473931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a2fd1c33-d45f-4330-9ad0-71d2cbe8d65d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:32:38.534470 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fd1c33-d45f-4330-9ad0-71d2cbe8d65d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:32:38.536081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fd1c33-d45f-4330-9ad0-71d2cbe8d65d service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:32:38.544973 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1205/2419] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:32:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:32:48.540820 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=28 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:02.098616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ec09a-ad89-4d49-a5a2-434c32e1014b service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:02.100057 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e43ec09a-ad89-4d49-a5a2-434c32e1014b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:02.248409 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ec09a-ad89-4d49-a5a2-434c32e1014b service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:02.256420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24089a-6fd5-4ff7-b8ed-49f4346bcd06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:02.257293 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24089a-6fd5-4ff7-b8ed-49f4346bcd06 service placement] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:02.257579 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24089a-6fd5-4ff7-b8ed-49f4346bcd06 service placement] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:02.257857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24089a-6fd5-4ff7-b8ed-49f4346bcd06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:02.260043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-3e24089a-6fd5-4ff7-b8ed-49f4346bcd06 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:02.274044 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24089a-6fd5-4ff7-b8ed-49f4346bcd06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:02.275267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24089a-6fd5-4ff7-b8ed-49f4346bcd06 service placement] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:02.292641 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1215/2420] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:33:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:33:12.239947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd1c33-d45f-4330-9ad0-71d2cbe8d65d service nova] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:12.241685 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a2fd1c33-d45f-4330-9ad0-71d2cbe8d65d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:12.273188 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd1c33-d45f-4330-9ad0-71d2cbe8d65d service nova] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:12.276838 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d538d-8d68-4147-807c-d58f3cf36237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:12.277492 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d538d-8d68-4147-807c-d58f3cf36237 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:12.277779 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d538d-8d68-4147-807c-d58f3cf36237 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:12.278054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d538d-8d68-4147-807c-d58f3cf36237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:12.279817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-536d538d-8d68-4147-807c-d58f3cf36237 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:12.282854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=63 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:12.291403 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d538d-8d68-4147-807c-d58f3cf36237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:12.292890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d538d-8d68-4147-807c-d58f3cf36237 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:12.301906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1206/2421] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:33:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:33:22.297800 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=15 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:34:56.525405 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24089a-6fd5-4ff7-b8ed-49f4346bcd06 service placement] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:34:56.546277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24089a-6fd5-4ff7-b8ed-49f4346bcd06 service placement] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:34:56.550128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b9a67-0811-4290-9705-e874dcccacfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:34:56.550735 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b9a67-0811-4290-9705-e874dcccacfc service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:56.551017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b9a67-0811-4290-9705-e874dcccacfc service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:56.551326 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b9a67-0811-4290-9705-e874dcccacfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:56.569557 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b9a67-0811-4290-9705-e874dcccacfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:34:56.570964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b9a67-0811-4290-9705-e874dcccacfc service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:34:56.594417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1216/2422] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:34:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:35:02.631721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d538d-8d68-4147-807c-d58f3cf36237 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:02.644245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d538d-8d68-4147-807c-d58f3cf36237 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:02.649824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb441f-2447-485e-b51b-23c814516bf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:02.650382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb441f-2447-485e-b51b-23c814516bf9 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:02.650709 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb441f-2447-485e-b51b-23c814516bf9 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:02.650920 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb441f-2447-485e-b51b-23c814516bf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:02.659105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb441f-2447-485e-b51b-23c814516bf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:02.660706 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb441f-2447-485e-b51b-23c814516bf9 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:02.690190 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1207/2423] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:35:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:35:06.590810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=17 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:35:12.355658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b9a67-0811-4290-9705-e874dcccacfc service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:12.375346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b9a67-0811-4290-9705-e874dcccacfc service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:12.378987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c3b84-b88a-4b76-9169-3186dfa0604f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:12.379614 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c3b84-b88a-4b76-9169-3186dfa0604f service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:12.379900 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c3b84-b88a-4b76-9169-3186dfa0604f service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:12.380245 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c3b84-b88a-4b76-9169-3186dfa0604f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:12.388939 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c3b84-b88a-4b76-9169-3186dfa0604f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:12.390456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c3b84-b88a-4b76-9169-3186dfa0604f service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:12.417282 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1217/2424] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:35:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:35:12.687887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=13 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:35:22.414626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=17 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:35:34.466421 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb441f-2447-485e-b51b-23c814516bf9 service nova] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:34.476307 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb441f-2447-485e-b51b-23c814516bf9 service nova] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:34.479891 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bd0fa-1479-4543-a7a7-620e76c0aebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:34.480490 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bd0fa-1479-4543-a7a7-620e76c0aebc service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:34.480771 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bd0fa-1479-4543-a7a7-620e76c0aebc service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:34.481045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bd0fa-1479-4543-a7a7-620e76c0aebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:34.491864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7bd0fa-1479-4543-a7a7-620e76c0aebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:34.493294 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7bd0fa-1479-4543-a7a7-620e76c0aebc service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:34.504210 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1208/2425] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:35:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:35:44.501023 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=6 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:36:01.800231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c3b84-b88a-4b76-9169-3186dfa0604f service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:36:01.812634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c3b84-b88a-4b76-9169-3186dfa0604f service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:36:01.816577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd16899-1691-4010-8b1e-da6b4ce3f08d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:36:01.817192 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd16899-1691-4010-8b1e-da6b4ce3f08d service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:36:01.817473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd16899-1691-4010-8b1e-da6b4ce3f08d service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:36:01.817748 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd16899-1691-4010-8b1e-da6b4ce3f08d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:36:01.826283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd16899-1691-4010-8b1e-da6b4ce3f08d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:36:01.827722 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd16899-1691-4010-8b1e-da6b4ce3f08d service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:36:01.835803 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1218/2426] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:36:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:36:11.832777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=6 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:37:12.550683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bd0fa-1479-4543-a7a7-620e76c0aebc service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:12.561520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bd0fa-1479-4543-a7a7-620e76c0aebc service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:12.565522 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875e7a2-4816-4e17-b8ec-eed560240e6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:12.566123 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875e7a2-4816-4e17-b8ec-eed560240e6c service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:12.566408 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875e7a2-4816-4e17-b8ec-eed560240e6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:12.566679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875e7a2-4816-4e17-b8ec-eed560240e6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:12.582086 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875e7a2-4816-4e17-b8ec-eed560240e6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:12.583549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875e7a2-4816-4e17-b8ec-eed560240e6c service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:12.613094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1209/2427] 10.222.0.78 () {60 vars in 1396 bytes} [Mon Sep 19 11:37:12 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:37:12.662662 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd16899-1691-4010-8b1e-da6b4ce3f08d service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:12.676811 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd16899-1691-4010-8b1e-da6b4ce3f08d service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:12.683386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2731c17-cc2c-45db-9d87-2769784c2f77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:12.684004 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2731c17-cc2c-45db-9d87-2769784c2f77 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:12.684387 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2731c17-cc2c-45db-9d87-2769784c2f77 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:12.684668 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2731c17-cc2c-45db-9d87-2769784c2f77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:12.693585 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2731c17-cc2c-45db-9d87-2769784c2f77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:12.695758 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2731c17-cc2c-45db-9d87-2769784c2f77 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:12.724434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1219/2428] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:37:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:37:22.609329 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=13 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:37:22.721562 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=13 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:38:04.056765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875e7a2-4816-4e17-b8ec-eed560240e6c service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:38:04.068363 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875e7a2-4816-4e17-b8ec-eed560240e6c service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:38:04.073727 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e58a0-62f0-420c-a9d0-89175a5b3ab8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:38:04.074328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e58a0-62f0-420c-a9d0-89175a5b3ab8 service placement] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:38:04.074612 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e58a0-62f0-420c-a9d0-89175a5b3ab8 service placement] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:38:04.074887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e58a0-62f0-420c-a9d0-89175a5b3ab8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:38:04.083849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e58a0-62f0-420c-a9d0-89175a5b3ab8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:38:04.085437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e58a0-62f0-420c-a9d0-89175a5b3ab8 service placement] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:38:04.096087 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1210/2429] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:38:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:38:12.611342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2731c17-cc2c-45db-9d87-2769784c2f77 service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:38:12.621608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2731c17-cc2c-45db-9d87-2769784c2f77 service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:38:12.625276 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0eaab-004f-4331-b4c6-4ecf14965f95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:38:12.625875 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0eaab-004f-4331-b4c6-4ecf14965f95 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:38:12.626155 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0eaab-004f-4331-b4c6-4ecf14965f95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:38:12.626429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0eaab-004f-4331-b4c6-4ecf14965f95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:38:12.634982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0eaab-004f-4331-b4c6-4ecf14965f95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:38:12.636424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0eaab-004f-4331-b4c6-4ecf14965f95 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:38:12.662839 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1220/2430] 10.222.0.78 () {60 vars in 1396 bytes} [Mon Sep 19 11:38:12 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:38:14.091770 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=6 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:38:22.660288 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=13 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:39:12.935180 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e58a0-62f0-420c-a9d0-89175a5b3ab8 service placement] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:39:12.946284 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e58a0-62f0-420c-a9d0-89175a5b3ab8 service placement] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:39:12.949998 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756464b7-a96c-487e-87f1-3296bf3a8af5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:39:12.950632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756464b7-a96c-487e-87f1-3296bf3a8af5 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:39:12.950917 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756464b7-a96c-487e-87f1-3296bf3a8af5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:39:12.951391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756464b7-a96c-487e-87f1-3296bf3a8af5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:39:12.961129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756464b7-a96c-487e-87f1-3296bf3a8af5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:39:12.962553 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756464b7-a96c-487e-87f1-3296bf3a8af5 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:39:12.991628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1211/2431] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:39:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:39:13.554614 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0eaab-004f-4331-b4c6-4ecf14965f95 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:39:13.567437 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0eaab-004f-4331-b4c6-4ecf14965f95 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:39:13.574362 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9129703-447e-4080-83c8-658069c4bb9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:39:13.575239 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9129703-447e-4080-83c8-658069c4bb9f service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:39:13.575547 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9129703-447e-4080-83c8-658069c4bb9f service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:39:13.575805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9129703-447e-4080-83c8-658069c4bb9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:39:13.579219 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b9129703-447e-4080-83c8-658069c4bb9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:39:13.605393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9129703-447e-4080-83c8-658069c4bb9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:39:13.607252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9129703-447e-4080-83c8-658069c4bb9f service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:39:13.621270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1221/2432] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:39:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:39:22.988988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=13 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:39:23.613973 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=11 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:40:12.958678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756464b7-a96c-487e-87f1-3296bf3a8af5 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:40:12.960598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-756464b7-a96c-487e-87f1-3296bf3a8af5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:40:13.031244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756464b7-a96c-487e-87f1-3296bf3a8af5 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:40:13.035213 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed3924-3592-4e16-b0b6-b9e9e39a8b91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:40:13.040848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed3924-3592-4e16-b0b6-b9e9e39a8b91 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:40:13.040848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed3924-3592-4e16-b0b6-b9e9e39a8b91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:40:13.040848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed3924-3592-4e16-b0b6-b9e9e39a8b91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:40:13.040848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c0ed3924-3592-4e16-b0b6-b9e9e39a8b91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:40:13.110713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed3924-3592-4e16-b0b6-b9e9e39a8b91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:40:13.112212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed3924-3592-4e16-b0b6-b9e9e39a8b91 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:40:13.124337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1212/2433] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:40:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:40:13.250915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9129703-447e-4080-83c8-658069c4bb9f service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:40:13.260147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9129703-447e-4080-83c8-658069c4bb9f service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:40:13.263786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3404e53-c59a-4771-8012-0b53f6222de7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:40:13.264381 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3404e53-c59a-4771-8012-0b53f6222de7 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:40:13.264661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3404e53-c59a-4771-8012-0b53f6222de7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:40:13.264966 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3404e53-c59a-4771-8012-0b53f6222de7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:40:13.266783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e3404e53-c59a-4771-8012-0b53f6222de7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:40:13.337611 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3404e53-c59a-4771-8012-0b53f6222de7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:40:13.339051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3404e53-c59a-4771-8012-0b53f6222de7 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:40:13.349694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1222/2434] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:40:13 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:40:13.491907 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed3924-3592-4e16-b0b6-b9e9e39a8b91 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:40:13.509373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed3924-3592-4e16-b0b6-b9e9e39a8b91 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:40:13.509373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b95be-3e84-401a-b84b-be7cf929661e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:40:13.509829 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b95be-3e84-401a-b84b-be7cf929661e service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:40:13.510098 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b95be-3e84-401a-b84b-be7cf929661e service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:40:13.510352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b95be-3e84-401a-b84b-be7cf929661e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:40:13.520066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5c9b95be-3e84-401a-b84b-be7cf929661e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:40:13.601277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b95be-3e84-401a-b84b-be7cf929661e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:40:13.602826 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b95be-3e84-401a-b84b-be7cf929661e service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:40:13.612546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1213/2435] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:40:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:40:23.344906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=33 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:40:23.607857 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=92 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:41:07.621332 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3404e53-c59a-4771-8012-0b53f6222de7 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:41:07.623030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e3404e53-c59a-4771-8012-0b53f6222de7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:41:07.693628 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3404e53-c59a-4771-8012-0b53f6222de7 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:41:07.702745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce07c89-3175-488b-8f5e-b740e8a8ae93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:41:07.703408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce07c89-3175-488b-8f5e-b740e8a8ae93 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:41:07.703689 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce07c89-3175-488b-8f5e-b740e8a8ae93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:41:07.703963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce07c89-3175-488b-8f5e-b740e8a8ae93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:41:07.705918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-dce07c89-3175-488b-8f5e-b740e8a8ae93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:41:07.716034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce07c89-3175-488b-8f5e-b740e8a8ae93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:41:07.721228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce07c89-3175-488b-8f5e-b740e8a8ae93 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:41:07.725320 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1223/2436] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:41:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:41:13.025584 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b95be-3e84-401a-b84b-be7cf929661e service nova] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:41:13.035532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b95be-3e84-401a-b84b-be7cf929661e service nova] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:41:13.039283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdc584-caf6-4bd6-9819-9befed03c6a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:41:13.039807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdc584-caf6-4bd6-9819-9befed03c6a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:41:13.040091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdc584-caf6-4bd6-9819-9befed03c6a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:41:13.040368 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdc584-caf6-4bd6-9819-9befed03c6a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:41:13.042301 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-bbcdc584-caf6-4bd6-9819-9befed03c6a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:41:13.056505 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcdc584-caf6-4bd6-9819-9befed03c6a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:41:13.057919 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcdc584-caf6-4bd6-9819-9befed03c6a1 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:41:13.069865 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1214/2437] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:41:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:41:13.374572 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce07c89-3175-488b-8f5e-b740e8a8ae93 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:41:13.387407 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce07c89-3175-488b-8f5e-b740e8a8ae93 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:41:13.394576 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609430e2-917a-4128-8555-f4d62531873d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:41:13.397901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609430e2-917a-4128-8555-f4d62531873d service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:41:13.397901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609430e2-917a-4128-8555-f4d62531873d service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:41:13.397901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609430e2-917a-4128-8555-f4d62531873d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:41:13.398426 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-609430e2-917a-4128-8555-f4d62531873d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:41:13.411740 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609430e2-917a-4128-8555-f4d62531873d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:41:13.413341 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609430e2-917a-4128-8555-f4d62531873d service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:41:13.433642 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1224/2438] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:41:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:41:23.064330 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=6 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:41:23.429179 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=37 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:42:13.621396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdc584-caf6-4bd6-9819-9befed03c6a1 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:42:13.632807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdc584-caf6-4bd6-9819-9befed03c6a1 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:42:13.638796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6b868-e7f8-463b-a908-74a018e3fd98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:42:13.641641 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6b868-e7f8-463b-a908-74a018e3fd98 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:13.641641 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6b868-e7f8-463b-a908-74a018e3fd98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:13.641641 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6b868-e7f8-463b-a908-74a018e3fd98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:13.641984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f0f6b868-e7f8-463b-a908-74a018e3fd98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:42:13.724371 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6b868-e7f8-463b-a908-74a018e3fd98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:42:13.726376 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6b868-e7f8-463b-a908-74a018e3fd98 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:42:13.736338 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1215/2439] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:42:13 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:42:14.029790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609430e2-917a-4128-8555-f4d62531873d service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:42:14.045441 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609430e2-917a-4128-8555-f4d62531873d service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:42:14.049944 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc5e44-ee90-46a6-b0af-020be7ad9a2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:42:14.050559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc5e44-ee90-46a6-b0af-020be7ad9a2b service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:42:14.050866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc5e44-ee90-46a6-b0af-020be7ad9a2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:42:14.051125 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc5e44-ee90-46a6-b0af-020be7ad9a2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:42:14.053169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6abc5e44-ee90-46a6-b0af-020be7ad9a2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:42:14.139277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc5e44-ee90-46a6-b0af-020be7ad9a2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:42:14.140712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc5e44-ee90-46a6-b0af-020be7ad9a2b service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:42:14.151373 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1225/2440] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:42:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:42:23.731509 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=33 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:42:24.146463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=33 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:07.155225 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6b868-e7f8-463b-a908-74a018e3fd98 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:07.156672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f0f6b868-e7f8-463b-a908-74a018e3fd98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:07.279379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6b868-e7f8-463b-a908-74a018e3fd98 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:07.284656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe6a8a-07e9-476a-bbe0-fc33358a88b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:07.285260 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe6a8a-07e9-476a-bbe0-fc33358a88b5 service placement] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:07.285549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe6a8a-07e9-476a-bbe0-fc33358a88b5 service placement] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:07.285826 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe6a8a-07e9-476a-bbe0-fc33358a88b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:07.287708 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d4fe6a8a-07e9-476a-bbe0-fc33358a88b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:07.299584 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe6a8a-07e9-476a-bbe0-fc33358a88b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:07.301167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe6a8a-07e9-476a-bbe0-fc33358a88b5 service placement] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:07.311980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1216/2441] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:43:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:43:14.243569 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc5e44-ee90-46a6-b0af-020be7ad9a2b service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:43:14.245826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6abc5e44-ee90-46a6-b0af-020be7ad9a2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:14.276225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc5e44-ee90-46a6-b0af-020be7ad9a2b service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:43:14.279940 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f0a30-a9ae-4275-8061-41e5fb5811cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:43:14.280571 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f0a30-a9ae-4275-8061-41e5fb5811cc service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:43:14.280864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f0a30-a9ae-4275-8061-41e5fb5811cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:43:14.281148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f0a30-a9ae-4275-8061-41e5fb5811cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:43:14.283054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-065f0a30-a9ae-4275-8061-41e5fb5811cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:43:14.354807 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f0a30-a9ae-4275-8061-41e5fb5811cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:43:14.356262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f0a30-a9ae-4275-8061-41e5fb5811cc service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:43:14.365183 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1226/2442] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:43:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:43:17.306797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=54 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:43:24.361092 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=42 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:44:13.546868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe6a8a-07e9-476a-bbe0-fc33358a88b5 service placement] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:44:13.548664 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d4fe6a8a-07e9-476a-bbe0-fc33358a88b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:13.583429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe6a8a-07e9-476a-bbe0-fc33358a88b5 service placement] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:44:13.588925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f0643-bea1-4c86-b2f8-1143966cbeb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:44:13.589525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f0643-bea1-4c86-b2f8-1143966cbeb6 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:44:13.589807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f0643-bea1-4c86-b2f8-1143966cbeb6 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:44:13.590089 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f0643-bea1-4c86-b2f8-1143966cbeb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:44:13.591993 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d01f0643-bea1-4c86-b2f8-1143966cbeb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:44:13.666351 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01f0643-bea1-4c86-b2f8-1143966cbeb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:44:13.668048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01f0643-bea1-4c86-b2f8-1143966cbeb6 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:44:13.679824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1217/2443] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:44:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:44:23.673220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=42 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:13.821333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f0a30-a9ae-4275-8061-41e5fb5811cc service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:13.822891 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-065f0a30-a9ae-4275-8061-41e5fb5811cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:45:13.855338 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f0a30-a9ae-4275-8061-41e5fb5811cc service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:13.860940 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdeb2f-accf-4384-abaf-df17cadce10b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:13.861551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdeb2f-accf-4384-abaf-df17cadce10b service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:13.861836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdeb2f-accf-4384-abaf-df17cadce10b service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:13.862111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdeb2f-accf-4384-abaf-df17cadce10b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:13.870783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bdeb2f-accf-4384-abaf-df17cadce10b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:13.872397 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bdeb2f-accf-4384-abaf-df17cadce10b service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:13.899982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1227/2444] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:45:13 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:45:13.954213 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f0643-bea1-4c86-b2f8-1143966cbeb6 service nova] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:13.973304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f0643-bea1-4c86-b2f8-1143966cbeb6 service nova] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:13.977042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f1f47-4dc5-4568-aaa2-ca0f22fdb2b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:13.977640 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f1f47-4dc5-4568-aaa2-ca0f22fdb2b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:13.977921 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f1f47-4dc5-4568-aaa2-ca0f22fdb2b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:13.978196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f1f47-4dc5-4568-aaa2-ca0f22fdb2b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:13.988596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f1f47-4dc5-4568-aaa2-ca0f22fdb2b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:13.990589 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f1f47-4dc5-4568-aaa2-ca0f22fdb2b1 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:14.001568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1218/2445] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:45:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:45:14.655978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdeb2f-accf-4384-abaf-df17cadce10b service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:45:14.675313 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdeb2f-accf-4384-abaf-df17cadce10b service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:45:14.678912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42599567-b597-4179-b858-25080f4a24ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:45:14.679890 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42599567-b597-4179-b858-25080f4a24ec service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:45:14.680172 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42599567-b597-4179-b858-25080f4a24ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:45:14.680446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42599567-b597-4179-b858-25080f4a24ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:45:14.689028 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42599567-b597-4179-b858-25080f4a24ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:45:14.690446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42599567-b597-4179-b858-25080f4a24ec service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:45:14.718039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1228/2446] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:45:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:45:23.996661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=10 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:45:24.715538 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=37 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:11.900223 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f1f47-4dc5-4568-aaa2-ca0f22fdb2b1 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:11.908945 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f1f47-4dc5-4568-aaa2-ca0f22fdb2b1 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:11.912615 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fbf5e-f684-4361-b0b7-604834d9c4de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:11.913213 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fbf5e-f684-4361-b0b7-604834d9c4de service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:11.913491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fbf5e-f684-4361-b0b7-604834d9c4de service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:11.913764 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fbf5e-f684-4361-b0b7-604834d9c4de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:11.924690 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fbf5e-f684-4361-b0b7-604834d9c4de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:11.926119 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fbf5e-f684-4361-b0b7-604834d9c4de service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:11.937943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1219/2447] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:46:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:46:14.776541 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42599567-b597-4179-b858-25080f4a24ec service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:46:14.786309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42599567-b597-4179-b858-25080f4a24ec service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:46:14.790712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb3b24-902c-466b-b2b8-28fa9e5e193d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:46:14.791371 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb3b24-902c-466b-b2b8-28fa9e5e193d service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:46:14.791633 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb3b24-902c-466b-b2b8-28fa9e5e193d service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:46:14.791903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb3b24-902c-466b-b2b8-28fa9e5e193d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:46:14.799454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb3b24-902c-466b-b2b8-28fa9e5e193d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:46:14.800869 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb3b24-902c-466b-b2b8-28fa9e5e193d service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:46:14.809728 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1229/2448] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:46:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:46:21.933094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=6 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:46:24.805523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=6 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:14.500205 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fbf5e-f684-4361-b0b7-604834d9c4de service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:14.501663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-129fbf5e-f684-4361-b0b7-604834d9c4de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:47:14.514237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fbf5e-f684-4361-b0b7-604834d9c4de service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:14.518012 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714131be-dfc1-44b0-81ce-03559e209569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:14.518627 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714131be-dfc1-44b0-81ce-03559e209569 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:14.518910 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714131be-dfc1-44b0-81ce-03559e209569 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:14.519197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714131be-dfc1-44b0-81ce-03559e209569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:14.520966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-714131be-dfc1-44b0-81ce-03559e209569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:47:14.605393 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714131be-dfc1-44b0-81ce-03559e209569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:14.606833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714131be-dfc1-44b0-81ce-03559e209569 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:14.625559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1220/2449] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:47:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:47:14.832745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb3b24-902c-466b-b2b8-28fa9e5e193d service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:14.842039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb3b24-902c-466b-b2b8-28fa9e5e193d service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:14.845670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bccfb3-7d4d-433c-9797-f9bed9f6501d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:14.846275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bccfb3-7d4d-433c-9797-f9bed9f6501d service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:14.846556 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bccfb3-7d4d-433c-9797-f9bed9f6501d service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:14.846832 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bccfb3-7d4d-433c-9797-f9bed9f6501d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:14.855623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bccfb3-7d4d-433c-9797-f9bed9f6501d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:14.857174 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bccfb3-7d4d-433c-9797-f9bed9f6501d service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:14.886717 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1230/2450] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:47:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:47:14.936421 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714131be-dfc1-44b0-81ce-03559e209569 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:14.946329 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714131be-dfc1-44b0-81ce-03559e209569 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:14.949854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd6673-4ad9-460a-86b7-7abc74857a44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:14.950449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd6673-4ad9-460a-86b7-7abc74857a44 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:14.950725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd6673-4ad9-460a-86b7-7abc74857a44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:14.950994 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd6673-4ad9-460a-86b7-7abc74857a44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:14.959944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd6673-4ad9-460a-86b7-7abc74857a44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:14.961374 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd6673-4ad9-460a-86b7-7abc74857a44 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:14.990273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1221/2451] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:47:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:47:18.043278 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bccfb3-7d4d-433c-9797-f9bed9f6501d service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:18.045437 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d7bccfb3-7d4d-433c-9797-f9bed9f6501d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:47:18.061012 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bccfb3-7d4d-433c-9797-f9bed9f6501d service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:18.066010 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44480a-52d0-4535-8326-5f394cb49fe9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:18.066646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44480a-52d0-4535-8326-5f394cb49fe9 service placement] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:18.066927 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44480a-52d0-4535-8326-5f394cb49fe9 service placement] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:18.067234 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44480a-52d0-4535-8326-5f394cb49fe9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:18.068994 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1b44480a-52d0-4535-8326-5f394cb49fe9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:47:18.081030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44480a-52d0-4535-8326-5f394cb49fe9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:18.082624 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44480a-52d0-4535-8326-5f394cb49fe9 service placement] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:18.092933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1231/2452] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:47:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:47:21.958691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70be8d-4216-4d05-9ead-ab8bf7979d52 None None] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:21.959308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70be8d-4216-4d05-9ead-ab8bf7979d52 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:21.959308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70be8d-4216-4d05-9ead-ab8bf7979d52 None None] PATH_INFO: `/` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:21.960631 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1222/2453] 10.222.0.78 () {54 vars in 921 bytes} [Mon Sep 19 11:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:47:21.993020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf8599-0bbe-42e5-808b-750da527c61a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:21.993568 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf8599-0bbe-42e5-808b-750da527c61a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:21.993568 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf8599-0bbe-42e5-808b-750da527c61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:22.052609 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-42cf8599-0bbe-42e5-808b-750da527c61a None None] MFA Rules not processed for user `0427cdd3905a422cab4fed0aac187ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:47:22.053843 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-42cf8599-0bbe-42e5-808b-750da527c61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:47:22.063415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1232/2454] 10.222.0.78 () {58 vars in 1018 bytes} [Mon Sep 19 11:47:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:47:22.074476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70be8d-4216-4d05-9ead-ab8bf7979d52 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:22.088828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70be8d-4216-4d05-9ead-ab8bf7979d52 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:22.094643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11563412-70c7-4266-9188-c143785f3f26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:22.102823 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11563412-70c7-4266-9188-c143785f3f26 service placement] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:22.102823 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11563412-70c7-4266-9188-c143785f3f26 service placement] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:22.102823 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11563412-70c7-4266-9188-c143785f3f26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:22.104332 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf8599-0bbe-42e5-808b-750da527c61a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:22.105896 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-42cf8599-0bbe-42e5-808b-750da527c61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:47:22.106095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11563412-70c7-4266-9188-c143785f3f26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:22.107849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11563412-70c7-4266-9188-c143785f3f26 service placement] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:22.120427 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1223/2455] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:47:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:47:22.130239 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf8599-0bbe-42e5-808b-750da527c61a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:22.136024 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99b87e-9ff9-4093-9061-ab7ebfec7112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:22.136733 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99b87e-9ff9-4093-9061-ab7ebfec7112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:22.137133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99b87e-9ff9-4093-9061-ab7ebfec7112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:22.137915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99b87e-9ff9-4093-9061-ab7ebfec7112 admin admin] PATH_INFO: `/v3/users/792178b0829f49c2b5729aeee1805e77` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:22.139292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99b87e-9ff9-4093-9061-ab7ebfec7112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=792178b0829f49c2b5729aeee1805e77)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:22.141262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99b87e-9ff9-4093-9061-ab7ebfec7112 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:22.274829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1d99b87e-9ff9-4093-9061-ab7ebfec7112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '792178b0829f49c2b5729aeee1805e77'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:22.286816 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1d99b87e-9ff9-4093-9061-ab7ebfec7112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '792178b0829f49c2b5729aeee1805e77'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:22.293806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1d99b87e-9ff9-4093-9061-ab7ebfec7112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '792178b0829f49c2b5729aeee1805e77'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:22.340319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1d99b87e-9ff9-4093-9061-ab7ebfec7112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '792178b0829f49c2b5729aeee1805e77'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:22.341025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1d99b87e-9ff9-4093-9061-ab7ebfec7112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '792178b0829f49c2b5729aeee1805e77'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:22.347187 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1233/2456] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:47:22 2022] DELETE /identity/v3/users/792178b0829f49c2b5729aeee1805e77 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:47:22.582449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11563412-70c7-4266-9188-c143785f3f26 service placement] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:22.597524 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11563412-70c7-4266-9188-c143785f3f26 service placement] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:22.656194 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e22ad-4359-4e95-9ee3-53382c986b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:22.656828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e22ad-4359-4e95-9ee3-53382c986b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:22.657110 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e22ad-4359-4e95-9ee3-53382c986b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:22.657386 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e22ad-4359-4e95-9ee3-53382c986b1d admin admin] PATH_INFO: `/v3/projects/4f82c67c47c243f594cd2bab7ac861aa` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:22.658499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e22ad-4359-4e95-9ee3-53382c986b1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f82c67c47c243f594cd2bab7ac861aa)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:22.660513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e22ad-4359-4e95-9ee3-53382c986b1d admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:22.754066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:22.770778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-966e22ad-4359-4e95-9ee3-53382c986b1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f82c67c47c243f594cd2bab7ac861aa', 'request_id': 'req-966e22ad-4359-4e95-9ee3-53382c986b1d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:22.776324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-966e22ad-4359-4e95-9ee3-53382c986b1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f82c67c47c243f594cd2bab7ac861aa', 'request_id': 'req-966e22ad-4359-4e95-9ee3-53382c986b1d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:22.866148 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-966e22ad-4359-4e95-9ee3-53382c986b1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f82c67c47c243f594cd2bab7ac861aa', 'request_id': 'req-966e22ad-4359-4e95-9ee3-53382c986b1d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:22.880732 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-966e22ad-4359-4e95-9ee3-53382c986b1d admin admin] The token cache is being invalidate because project 4f82c67c47c243f594cd2bab7ac861aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:47:22.883035 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-966e22ad-4359-4e95-9ee3-53382c986b1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:22.883035 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-966e22ad-4359-4e95-9ee3-53382c986b1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:22.884063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1224/2457] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:47:22 2022] DELETE /identity/v3/projects/4f82c67c47c243f594cd2bab7ac861aa => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:47:22.888286 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99b87e-9ff9-4093-9061-ab7ebfec7112 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:22.889733 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1d99b87e-9ff9-4093-9061-ab7ebfec7112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:47:22.904677 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99b87e-9ff9-4093-9061-ab7ebfec7112 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:22.951537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:22.960111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62f531-50b2-431f-9fd7-292c4bc7818e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:22.960747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62f531-50b2-431f-9fd7-292c4bc7818e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:22.966827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62f531-50b2-431f-9fd7-292c4bc7818e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:22.966827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62f531-50b2-431f-9fd7-292c4bc7818e admin admin] PATH_INFO: `/v3/users/f9e2834bd72946aa8d6655d2937f7eb4` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:22.976348 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62f531-50b2-431f-9fd7-292c4bc7818e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9e2834bd72946aa8d6655d2937f7eb4)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:22.978311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62f531-50b2-431f-9fd7-292c4bc7818e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:23.179416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bb62f531-50b2-431f-9fd7-292c4bc7818e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9e2834bd72946aa8d6655d2937f7eb4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:23.193086 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bb62f531-50b2-431f-9fd7-292c4bc7818e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9e2834bd72946aa8d6655d2937f7eb4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:23.200737 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bb62f531-50b2-431f-9fd7-292c4bc7818e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9e2834bd72946aa8d6655d2937f7eb4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:23.277274 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bb62f531-50b2-431f-9fd7-292c4bc7818e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9e2834bd72946aa8d6655d2937f7eb4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:23.278559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bb62f531-50b2-431f-9fd7-292c4bc7818e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9e2834bd72946aa8d6655d2937f7eb4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:23.285239 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1234/2458] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:47:22 2022] DELETE /identity/v3/users/f9e2834bd72946aa8d6655d2937f7eb4 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:47:23.525191 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e22ad-4359-4e95-9ee3-53382c986b1d admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:47:23.526661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-966e22ad-4359-4e95-9ee3-53382c986b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:47:23.545067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e22ad-4359-4e95-9ee3-53382c986b1d admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:47:23.726046 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8204d-2999-4d2f-957d-705330975633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:47:23.726700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8204d-2999-4d2f-957d-705330975633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:47:23.726975 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8204d-2999-4d2f-957d-705330975633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:47:23.727273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8204d-2999-4d2f-957d-705330975633 admin admin] PATH_INFO: `/v3/projects/cd13f6e839454a7792487439e155534c` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:47:23.738266 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8204d-2999-4d2f-957d-705330975633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd13f6e839454a7792487439e155534c)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:47:23.740382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8204d-2999-4d2f-957d-705330975633 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:47:23.878238 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-c2b8204d-2999-4d2f-957d-705330975633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd13f6e839454a7792487439e155534c', 'request_id': 'req-c2b8204d-2999-4d2f-957d-705330975633'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:23.883978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-c2b8204d-2999-4d2f-957d-705330975633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd13f6e839454a7792487439e155534c', 'request_id': 'req-c2b8204d-2999-4d2f-957d-705330975633'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:23.957438 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-c2b8204d-2999-4d2f-957d-705330975633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd13f6e839454a7792487439e155534c', 'request_id': 'req-c2b8204d-2999-4d2f-957d-705330975633'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:23.970068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-c2b8204d-2999-4d2f-957d-705330975633 admin admin] The token cache is being invalidate because project cd13f6e839454a7792487439e155534c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:47:23.970427 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-c2b8204d-2999-4d2f-957d-705330975633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:23.971002 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-c2b8204d-2999-4d2f-957d-705330975633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:47:23.972537 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1225/2459] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:47:23 2022] DELETE /identity/v3/projects/cd13f6e839454a7792487439e155534c => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:47:33.281416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=34,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:47:33.967564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:DELETE=13,keystone:SELECT=69,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:48:11.056442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62f531-50b2-431f-9fd7-292c4bc7818e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:48:11.057901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bb62f531-50b2-431f-9fd7-292c4bc7818e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:48:11.135955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62f531-50b2-431f-9fd7-292c4bc7818e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:48:11.200432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac5b8d-4617-4b74-9131-67a7bf2e49d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:48:11.201055 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac5b8d-4617-4b74-9131-67a7bf2e49d3 service placement] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:48:11.201335 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac5b8d-4617-4b74-9131-67a7bf2e49d3 service placement] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:48:11.201608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac5b8d-4617-4b74-9131-67a7bf2e49d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:48:11.203443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-78ac5b8d-4617-4b74-9131-67a7bf2e49d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:48:11.214951 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac5b8d-4617-4b74-9131-67a7bf2e49d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:48:11.216866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac5b8d-4617-4b74-9131-67a7bf2e49d3 service placement] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:48:11.227865 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1235/2460] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:48:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:48:14.480821 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8204d-2999-4d2f-957d-705330975633 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:48:14.480821 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c2b8204d-2999-4d2f-957d-705330975633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:48:14.538845 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8204d-2999-4d2f-957d-705330975633 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:48:14.542474 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1447c-2f59-4308-a4e5-c0f251821539 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:48:14.543080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1447c-2f59-4308-a4e5-c0f251821539 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:48:14.543393 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1447c-2f59-4308-a4e5-c0f251821539 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:48:14.543681 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1447c-2f59-4308-a4e5-c0f251821539 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:48:14.545438 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-15f1447c-2f59-4308-a4e5-c0f251821539 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:48:14.617532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1447c-2f59-4308-a4e5-c0f251821539 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:48:14.618946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1447c-2f59-4308-a4e5-c0f251821539 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:48:14.637560 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1226/2461] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:48:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:48:21.223389 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=58 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:48:24.634970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=56 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:14.835076 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac5b8d-4617-4b74-9131-67a7bf2e49d3 service placement] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:14.844944 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac5b8d-4617-4b74-9131-67a7bf2e49d3 service placement] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:14.848517 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb669e30-a239-4e86-886f-8f0ec3bc1851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:14.849106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb669e30-a239-4e86-886f-8f0ec3bc1851 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:14.849383 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb669e30-a239-4e86-886f-8f0ec3bc1851 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:14.849651 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb669e30-a239-4e86-886f-8f0ec3bc1851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:14.851397 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fb669e30-a239-4e86-886f-8f0ec3bc1851 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:14.922033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb669e30-a239-4e86-886f-8f0ec3bc1851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:14.923503 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb669e30-a239-4e86-886f-8f0ec3bc1851 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:14.932549 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1236/2462] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:50:15.097156 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1447c-2f59-4308-a4e5-c0f251821539 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:15.105654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1447c-2f59-4308-a4e5-c0f251821539 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:15.109319 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b57f3b-a6e1-4528-b7b4-7333108cc4d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:15.109934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b57f3b-a6e1-4528-b7b4-7333108cc4d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:15.110195 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b57f3b-a6e1-4528-b7b4-7333108cc4d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:15.110461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b57f3b-a6e1-4528-b7b4-7333108cc4d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:15.112377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-57b57f3b-a6e1-4528-b7b4-7333108cc4d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:15.182891 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b57f3b-a6e1-4528-b7b4-7333108cc4d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:15.184356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b57f3b-a6e1-4528-b7b4-7333108cc4d1 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:15.193428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1227/2463] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:50:15.236521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb669e30-a239-4e86-886f-8f0ec3bc1851 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:15.245567 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb669e30-a239-4e86-886f-8f0ec3bc1851 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:15.249314 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecd11c-ba8a-416b-b3d8-f292c9dd2708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:15.249919 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecd11c-ba8a-416b-b3d8-f292c9dd2708 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:15.250202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecd11c-ba8a-416b-b3d8-f292c9dd2708 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:15.250478 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecd11c-ba8a-416b-b3d8-f292c9dd2708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:15.252268 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-64ecd11c-ba8a-416b-b3d8-f292c9dd2708 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:15.280693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-64ecd11c-ba8a-416b-b3d8-f292c9dd2708 service cinder] Could not find project: 4f82c67c47c243f594cd2bab7ac861aa.: keystone.exception.ProjectNotFound: Could not find project: 4f82c67c47c243f594cd2bab7ac861aa. Sep 19 11:50:15.280693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:50:15.280693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:50:15.280693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:50:15.280693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:50:15.280693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:50:15.280693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:50:15.280693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:50:15.280693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:50:15.280693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:50:15.280693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:50:15.280693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:50:15.280693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 19 11:50:15.280693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 19 11:50:15.280693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 19 11:50:15.281963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 19 11:50:15.281963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 19 11:50:15.281963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 19 11:50:15.281963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:50:15.281963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:50:15.281963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 19 11:50:15.281963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 19 11:50:15.281963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:50:15.281963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:50:15.281963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:50:15.281963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:50:15.281963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:50:15.281963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:50:15.281963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:50:15.281963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:50:15.281963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:50:15.281963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:50:15.283214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:50:15.283214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:50:15.283214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:50:15.283214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:50:15.283214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 19 11:50:15.283214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 19 11:50:15.283214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 19 11:50:15.283214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 19 11:50:15.283214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 19 11:50:15.283214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 19 11:50:15.283214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 19 11:50:15.283214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 19 11:50:15.283214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:50:15.283214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:50:15.283214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:50:15.283214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:50:15.283214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:50:15.284419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:50:15.284419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:50:15.284419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:50:15.284419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:50:15.284419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:50:15.284419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:50:15.284419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:50:15.284419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:50:15.284419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:50:15.284419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:50:15.284419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:50:15.284419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:50:15.284419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:50:15.284419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:50:15.284419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:50:15.284419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:50:15.284419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:50:15.285652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 4f82c67c47c243f594cd2bab7ac861aa. Sep 19 11:50:15.285652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:50:15.285652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1237/2464] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:15 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 19 11:50:15.983283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b57f3b-a6e1-4528-b7b4-7333108cc4d1 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:50:15.984822 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-57b57f3b-a6e1-4528-b7b4-7333108cc4d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:16.044749 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b57f3b-a6e1-4528-b7b4-7333108cc4d1 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:50:16.051579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6971e-80f9-4c70-be50-b6cdbaeb18db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:50:16.052296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6971e-80f9-4c70-be50-b6cdbaeb18db service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:50:16.052508 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6971e-80f9-4c70-be50-b6cdbaeb18db service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:50:16.052780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6971e-80f9-4c70-be50-b6cdbaeb18db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:50:16.057707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b2b6971e-80f9-4c70-be50-b6cdbaeb18db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:50:16.148512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6971e-80f9-4c70-be50-b6cdbaeb18db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:50:16.149957 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6971e-80f9-4c70-be50-b6cdbaeb18db service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:50:16.169131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1228/2465] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:50:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:50:25.256291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=37 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:50:26.157592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=92 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:12.394020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecd11c-ba8a-416b-b3d8-f292c9dd2708 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:12.405377 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecd11c-ba8a-416b-b3d8-f292c9dd2708 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:12.409331 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65682304-de46-48e7-8817-27126923bd75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:12.410020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65682304-de46-48e7-8817-27126923bd75 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:12.410311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65682304-de46-48e7-8817-27126923bd75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:12.410643 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65682304-de46-48e7-8817-27126923bd75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:12.412444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-65682304-de46-48e7-8817-27126923bd75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:12.427788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65682304-de46-48e7-8817-27126923bd75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:12.429222 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65682304-de46-48e7-8817-27126923bd75 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:12.441200 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1238/2466] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:22.435977 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=6 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:51:23.779782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6971e-80f9-4c70-be50-b6cdbaeb18db service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:51:23.788570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6971e-80f9-4c70-be50-b6cdbaeb18db service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:51:23.792675 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f95d0-3032-4db7-9af5-02f3157631c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:51:23.793317 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f95d0-3032-4db7-9af5-02f3157631c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:51:23.793598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f95d0-3032-4db7-9af5-02f3157631c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:51:23.793938 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f95d0-3032-4db7-9af5-02f3157631c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:51:23.795802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-905f95d0-3032-4db7-9af5-02f3157631c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:51:23.818863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f95d0-3032-4db7-9af5-02f3157631c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:51:23.820331 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f95d0-3032-4db7-9af5-02f3157631c1 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:51:23.828971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1229/2467] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:51:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:51:33.825183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=11 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:15.023000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65682304-de46-48e7-8817-27126923bd75 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:52:15.031815 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65682304-de46-48e7-8817-27126923bd75 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:52:15.035819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc9930-52df-4d8f-b8db-b93eb4de8105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:52:15.036443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc9930-52df-4d8f-b8db-b93eb4de8105 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:15.036717 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc9930-52df-4d8f-b8db-b93eb4de8105 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:15.037011 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc9930-52df-4d8f-b8db-b93eb4de8105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:15.038821 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c6fc9930-52df-4d8f-b8db-b93eb4de8105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:15.109566 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc9930-52df-4d8f-b8db-b93eb4de8105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:52:15.111011 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc9930-52df-4d8f-b8db-b93eb4de8105 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:52:15.125585 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1239/2468] 10.222.0.78 () {60 vars in 1396 bytes} [Mon Sep 19 11:52:15 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:52:15.465560 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f95d0-3032-4db7-9af5-02f3157631c1 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:52:15.467060 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-905f95d0-3032-4db7-9af5-02f3157631c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:15.483245 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f95d0-3032-4db7-9af5-02f3157631c1 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:52:15.488747 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9dffd-808e-4689-8f3d-3d1ab52f7ed8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:52:15.489364 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9dffd-808e-4689-8f3d-3d1ab52f7ed8 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:15.489637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9dffd-808e-4689-8f3d-3d1ab52f7ed8 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:15.489908 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9dffd-808e-4689-8f3d-3d1ab52f7ed8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:15.491769 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e4a9dffd-808e-4689-8f3d-3d1ab52f7ed8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:15.578276 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a9dffd-808e-4689-8f3d-3d1ab52f7ed8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:52:15.578276 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a9dffd-808e-4689-8f3d-3d1ab52f7ed8 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:52:15.585566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1230/2469] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:52:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:52:17.024706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc9930-52df-4d8f-b8db-b93eb4de8105 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:52:17.024706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c6fc9930-52df-4d8f-b8db-b93eb4de8105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:17.043852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc9930-52df-4d8f-b8db-b93eb4de8105 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:52:17.049756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee10df-0860-4537-bf53-70d6d5337f56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:52:17.050449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee10df-0860-4537-bf53-70d6d5337f56 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:52:17.050734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee10df-0860-4537-bf53-70d6d5337f56 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:52:17.051009 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee10df-0860-4537-bf53-70d6d5337f56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:52:17.052978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-03ee10df-0860-4537-bf53-70d6d5337f56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:52:17.066543 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee10df-0860-4537-bf53-70d6d5337f56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:52:17.068197 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee10df-0860-4537-bf53-70d6d5337f56 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:52:17.080487 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1240/2470] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:52:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:52:25.580736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=33 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:52:27.074972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=39 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:53:14.663674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9dffd-808e-4689-8f3d-3d1ab52f7ed8 service nova] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:53:14.694221 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9dffd-808e-4689-8f3d-3d1ab52f7ed8 service nova] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:53:14.699623 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10653a16-66fa-4820-9432-3a6b3fd940f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:53:14.700211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10653a16-66fa-4820-9432-3a6b3fd940f6 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:14.700487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10653a16-66fa-4820-9432-3a6b3fd940f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:14.700756 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10653a16-66fa-4820-9432-3a6b3fd940f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:14.708279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10653a16-66fa-4820-9432-3a6b3fd940f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:53:14.709831 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10653a16-66fa-4820-9432-3a6b3fd940f6 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:53:14.719017 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1231/2471] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:53:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:53:18.182218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee10df-0860-4537-bf53-70d6d5337f56 service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:53:18.183646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-03ee10df-0860-4537-bf53-70d6d5337f56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:53:18.257853 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee10df-0860-4537-bf53-70d6d5337f56 service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:53:18.263908 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9edd92-c45a-44b2-8901-46d8292f82f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:53:18.264463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9edd92-c45a-44b2-8901-46d8292f82f9 service placement] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:53:18.264757 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9edd92-c45a-44b2-8901-46d8292f82f9 service placement] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:53:18.265025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9edd92-c45a-44b2-8901-46d8292f82f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:53:18.274757 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9edd92-c45a-44b2-8901-46d8292f82f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:53:18.276501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9edd92-c45a-44b2-8901-46d8292f82f9 service placement] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:53:18.287874 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1241/2472] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:53:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:53:24.714497 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=15 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:53:28.283810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=28 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:55:15.778254 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10653a16-66fa-4820-9432-3a6b3fd940f6 service nova] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:55:15.779718 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-10653a16-66fa-4820-9432-3a6b3fd940f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:55:15.833082 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10653a16-66fa-4820-9432-3a6b3fd940f6 service nova] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:55:15.837268 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff565d-4388-4c67-9e16-9a2300d8aaf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:55:15.837867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff565d-4388-4c67-9e16-9a2300d8aaf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:15.838142 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff565d-4388-4c67-9e16-9a2300d8aaf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:15.838416 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff565d-4388-4c67-9e16-9a2300d8aaf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:15.846918 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff565d-4388-4c67-9e16-9a2300d8aaf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:55:15.848513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff565d-4388-4c67-9e16-9a2300d8aaf2 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:55:15.876762 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1232/2473] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:55:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:55:15.920029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9edd92-c45a-44b2-8901-46d8292f82f9 service placement] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:55:15.939156 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9edd92-c45a-44b2-8901-46d8292f82f9 service placement] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:55:15.942999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc332395-2bf2-457b-b264-4efd5c610dc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:55:15.943629 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc332395-2bf2-457b-b264-4efd5c610dc9 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:15.943910 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc332395-2bf2-457b-b264-4efd5c610dc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:15.944767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc332395-2bf2-457b-b264-4efd5c610dc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:15.953339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc332395-2bf2-457b-b264-4efd5c610dc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:55:15.954747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc332395-2bf2-457b-b264-4efd5c610dc9 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:55:15.981757 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1242/2474] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:55:15 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:55:16.225919 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff565d-4388-4c67-9e16-9a2300d8aaf2 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:55:16.245368 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff565d-4388-4c67-9e16-9a2300d8aaf2 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:55:16.249132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b80b83-31dc-4782-aa83-87fe7496325c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:55:16.249745 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b80b83-31dc-4782-aa83-87fe7496325c service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:55:16.250024 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b80b83-31dc-4782-aa83-87fe7496325c service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:55:16.250325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b80b83-31dc-4782-aa83-87fe7496325c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:55:16.268050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b80b83-31dc-4782-aa83-87fe7496325c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:55:16.269580 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b80b83-31dc-4782-aa83-87fe7496325c service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:55:16.292695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1233/2475] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:55:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:55:25.979208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=18 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:55:26.289807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=48 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:56:12.302850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc332395-2bf2-457b-b264-4efd5c610dc9 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:56:12.312551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc332395-2bf2-457b-b264-4efd5c610dc9 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:56:12.316269 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df34ff-f4dc-4bcc-bb93-3dd61d132a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:56:12.317605 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df34ff-f4dc-4bcc-bb93-3dd61d132a86 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:56:12.317889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df34ff-f4dc-4bcc-bb93-3dd61d132a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:56:12.318168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df34ff-f4dc-4bcc-bb93-3dd61d132a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:56:12.332398 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df34ff-f4dc-4bcc-bb93-3dd61d132a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:56:12.333835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df34ff-f4dc-4bcc-bb93-3dd61d132a86 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:56:12.345074 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1243/2476] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:56:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:56:16.956869 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71914d-aff4-490e-9a0f-6c5379e17f06 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:56:16.957373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71914d-aff4-490e-9a0f-6c5379e17f06 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:56:16.957373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71914d-aff4-490e-9a0f-6c5379e17f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:56:17.016027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-af71914d-aff4-490e-9a0f-6c5379e17f06 None None] MFA Rules not processed for user `5a667fa07b1148998959835d9995ad60`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:56:17.017480 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-af71914d-aff4-490e-9a0f-6c5379e17f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:56:17.103869 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1234/2477] 10.222.0.78 () {58 vars in 967 bytes} [Mon Sep 19 11:56:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:56:17.273160 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df34ff-f4dc-4bcc-bb93-3dd61d132a86 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:56:17.275630 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-66df34ff-f4dc-4bcc-bb93-3dd61d132a86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:56:17.381288 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df34ff-f4dc-4bcc-bb93-3dd61d132a86 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:56:17.386446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77815ae-62df-47f0-be24-9bdceaa3796c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf462b2aa3fc4e2ca58851549cf0c644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:56:17.387067 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77815ae-62df-47f0-be24-9bdceaa3796c service swift] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:56:17.388649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77815ae-62df-47f0-be24-9bdceaa3796c service swift] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:56:17.388649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77815ae-62df-47f0-be24-9bdceaa3796c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:56:17.397788 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77815ae-62df-47f0-be24-9bdceaa3796c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:56:17.399432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77815ae-62df-47f0-be24-9bdceaa3796c service swift] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:56:17.408353 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1244/2478] 10.222.0.78 () {60 vars in 1384 bytes} [Mon Sep 19 11:56:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:56:17.515704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71914d-aff4-490e-9a0f-6c5379e17f06 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:56:17.530849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71914d-aff4-490e-9a0f-6c5379e17f06 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:56:17.540299 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca574a-537d-485b-89ff-ade58969f9d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:56:17.540917 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca574a-537d-485b-89ff-ade58969f9d9 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:56:17.541251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca574a-537d-485b-89ff-ade58969f9d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:56:17.541586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca574a-537d-485b-89ff-ade58969f9d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:56:17.544343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5aca574a-537d-485b-89ff-ade58969f9d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:56:17.560515 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca574a-537d-485b-89ff-ade58969f9d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:56:17.562512 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca574a-537d-485b-89ff-ade58969f9d9 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:56:17.581480 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1235/2479] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:56:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:56:18.789909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77815ae-62df-47f0-be24-9bdceaa3796c service swift] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:56:18.791595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d77815ae-62df-47f0-be24-9bdceaa3796c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:56:18.807413 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77815ae-62df-47f0-be24-9bdceaa3796c service swift] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:56:18.811142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b5864-80c1-4404-a97d-1e8129f24674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:56:18.811776 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b5864-80c1-4404-a97d-1e8129f24674 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:56:18.812049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b5864-80c1-4404-a97d-1e8129f24674 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:56:18.812319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b5864-80c1-4404-a97d-1e8129f24674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:56:18.814120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f28b5864-80c1-4404-a97d-1e8129f24674 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:56:18.879693 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28b5864-80c1-4404-a97d-1e8129f24674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:56:18.881204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28b5864-80c1-4404-a97d-1e8129f24674 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:56:18.891512 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1245/2480] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:56:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:56:20.372204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473d19e-8695-441e-8027-077e875a6a47 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:56:20.372204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473d19e-8695-441e-8027-077e875a6a47 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:56:20.372204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473d19e-8695-441e-8027-077e875a6a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:56:20.405560 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-4473d19e-8695-441e-8027-077e875a6a47 None None] MFA Rules not processed for user `5a8fb193bdd1448eab774bd1eef437c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:56:20.407785 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4473d19e-8695-441e-8027-077e875a6a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:56:20.422624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1236/2481] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:56:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:56:20.763036 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b5864-80c1-4404-a97d-1e8129f24674 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:56:20.772922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b5864-80c1-4404-a97d-1e8129f24674 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:56:20.776915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc01718-2276-4dfb-bdd8-5a432aad5575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:56:20.777560 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc01718-2276-4dfb-bdd8-5a432aad5575 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:56:20.777846 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc01718-2276-4dfb-bdd8-5a432aad5575 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:56:20.778152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc01718-2276-4dfb-bdd8-5a432aad5575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:56:20.780007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-cfc01718-2276-4dfb-bdd8-5a432aad5575 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:56:20.790413 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc01718-2276-4dfb-bdd8-5a432aad5575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:56:20.792154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc01718-2276-4dfb-bdd8-5a432aad5575 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:56:20.801131 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1246/2482] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:56:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:56:21.285966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473d19e-8695-441e-8027-077e875a6a47 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:56:21.288471 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4473d19e-8695-441e-8027-077e875a6a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:56:21.328365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473d19e-8695-441e-8027-077e875a6a47 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:56:21.334669 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1463ff3-a790-4ff5-8f70-557e60f50383 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:56:21.335438 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1463ff3-a790-4ff5-8f70-557e60f50383 service placement] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:56:21.335717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1463ff3-a790-4ff5-8f70-557e60f50383 service placement] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:56:21.335989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1463ff3-a790-4ff5-8f70-557e60f50383 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:56:21.337775 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c1463ff3-a790-4ff5-8f70-557e60f50383 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:56:21.350752 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1463ff3-a790-4ff5-8f70-557e60f50383 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:56:21.352494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1463ff3-a790-4ff5-8f70-557e60f50383 service placement] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:56:21.362667 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1237/2483] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:56:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:56:22.274192 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc01718-2276-4dfb-bdd8-5a432aad5575 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:56:22.287712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc01718-2276-4dfb-bdd8-5a432aad5575 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:56:22.292832 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11980e-7ed1-485f-bcf9-5a32d6559fcb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:56:22.293444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11980e-7ed1-485f-bcf9-5a32d6559fcb service placement] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:56:22.293724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11980e-7ed1-485f-bcf9-5a32d6559fcb service placement] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:56:22.294414 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11980e-7ed1-485f-bcf9-5a32d6559fcb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:56:22.296275 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2e11980e-7ed1-485f-bcf9-5a32d6559fcb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:56:22.309447 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11980e-7ed1-485f-bcf9-5a32d6559fcb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:56:22.311033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11980e-7ed1-485f-bcf9-5a32d6559fcb service placement] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:56:22.321701 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1247/2484] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:56:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:56:24.747357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1463ff3-a790-4ff5-8f70-557e60f50383 service placement] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:56:24.760874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1463ff3-a790-4ff5-8f70-557e60f50383 service placement] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:56:24.769369 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff18623-4d67-4aef-9c92-56e041b10245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:56:24.770294 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff18623-4d67-4aef-9c92-56e041b10245 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:56:24.770833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff18623-4d67-4aef-9c92-56e041b10245 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:56:24.771317 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff18623-4d67-4aef-9c92-56e041b10245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:56:24.796556 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff18623-4d67-4aef-9c92-56e041b10245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:56:24.798016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff18623-4d67-4aef-9c92-56e041b10245 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:56:24.810159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1238/2485] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:56:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:56:26.392938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11980e-7ed1-485f-bcf9-5a32d6559fcb service placement] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:56:26.404952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11980e-7ed1-485f-bcf9-5a32d6559fcb service placement] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:56:26.412886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95920bb5-3395-4840-b105-1c448b2b7714 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:56:26.413507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95920bb5-3395-4840-b105-1c448b2b7714 service placement] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:56:26.413907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95920bb5-3395-4840-b105-1c448b2b7714 service placement] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:56:26.414210 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95920bb5-3395-4840-b105-1c448b2b7714 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:56:26.416402 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-95920bb5-3395-4840-b105-1c448b2b7714 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:56:26.429696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95920bb5-3395-4840-b105-1c448b2b7714 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:56:26.433501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95920bb5-3395-4840-b105-1c448b2b7714 service placement] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:56:26.442359 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1248/2486] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:56:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:56:26.522789 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff18623-4d67-4aef-9c92-56e041b10245 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:56:26.524267 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1ff18623-4d67-4aef-9c92-56e041b10245 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:56:26.540945 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff18623-4d67-4aef-9c92-56e041b10245 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:56:26.545732 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21cfd03-f2fd-4b3b-a8ee-24059582ce2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:56:26.546361 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cfd03-f2fd-4b3b-a8ee-24059582ce2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:56:26.546640 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cfd03-f2fd-4b3b-a8ee-24059582ce2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:56:26.546914 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cfd03-f2fd-4b3b-a8ee-24059582ce2a admin admin] PATH_INFO: `/v3/users/28638075e935490bb7c8c229de9d14c5` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:56:26.548072 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21cfd03-f2fd-4b3b-a8ee-24059582ce2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=28638075e935490bb7c8c229de9d14c5)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:56:26.550096 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21cfd03-f2fd-4b3b-a8ee-24059582ce2a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:56:26.736203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-c21cfd03-f2fd-4b3b-a8ee-24059582ce2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28638075e935490bb7c8c229de9d14c5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:26.749074 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-c21cfd03-f2fd-4b3b-a8ee-24059582ce2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28638075e935490bb7c8c229de9d14c5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:26.756029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-c21cfd03-f2fd-4b3b-a8ee-24059582ce2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28638075e935490bb7c8c229de9d14c5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:26.757679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:56:26.824292 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-c21cfd03-f2fd-4b3b-a8ee-24059582ce2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28638075e935490bb7c8c229de9d14c5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:26.824930 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-c21cfd03-f2fd-4b3b-a8ee-24059582ce2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28638075e935490bb7c8c229de9d14c5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:26.831323 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1239/2487] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:56:26 2022] DELETE /identity/v3/users/28638075e935490bb7c8c229de9d14c5 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:56:27.111691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95920bb5-3395-4840-b105-1c448b2b7714 service placement] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:56:27.122425 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95920bb5-3395-4840-b105-1c448b2b7714 service placement] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:56:27.133069 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:56:27.183807 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:56:27.184451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:56:27.184732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:56:27.185009 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c admin admin] PATH_INFO: `/v3/projects/ec8ddabe0fd049de97da2f63453aabef` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:56:27.186140 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec8ddabe0fd049de97da2f63453aabef)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:56:27.188104 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:56:27.334248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec8ddabe0fd049de97da2f63453aabef', 'request_id': 'req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:27.415973 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec8ddabe0fd049de97da2f63453aabef', 'request_id': 'req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:27.423460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec8ddabe0fd049de97da2f63453aabef', 'request_id': 'req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:27.435089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c admin admin] The token cache is being invalidate because project ec8ddabe0fd049de97da2f63453aabef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:56:27.435498 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:27.436230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:27.437798 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1249/2488] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:56:27 2022] DELETE /identity/v3/projects/ec8ddabe0fd049de97da2f63453aabef => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:56:27.442428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21cfd03-f2fd-4b3b-a8ee-24059582ce2a admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:56:27.444131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c21cfd03-f2fd-4b3b-a8ee-24059582ce2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:56:27.458441 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21cfd03-f2fd-4b3b-a8ee-24059582ce2a admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:56:27.515314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689b2a4-38ce-4183-a94c-91a6e1faddac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:56:27.515951 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689b2a4-38ce-4183-a94c-91a6e1faddac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:56:27.516237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689b2a4-38ce-4183-a94c-91a6e1faddac admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:56:27.516513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689b2a4-38ce-4183-a94c-91a6e1faddac admin admin] PATH_INFO: `/v3/users/254d381deace4007a67ecb9ee7b6144b` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:56:27.531821 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689b2a4-38ce-4183-a94c-91a6e1faddac admin admin] RBAC: Authorizing `identity:delete_user(user_id=254d381deace4007a67ecb9ee7b6144b)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:56:27.533766 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689b2a4-38ce-4183-a94c-91a6e1faddac admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:56:27.693355 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-d689b2a4-38ce-4183-a94c-91a6e1faddac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '254d381deace4007a67ecb9ee7b6144b'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:27.705904 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-d689b2a4-38ce-4183-a94c-91a6e1faddac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '254d381deace4007a67ecb9ee7b6144b'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:27.713996 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-d689b2a4-38ce-4183-a94c-91a6e1faddac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '254d381deace4007a67ecb9ee7b6144b'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:27.775899 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-d689b2a4-38ce-4183-a94c-91a6e1faddac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '254d381deace4007a67ecb9ee7b6144b'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:27.776596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-d689b2a4-38ce-4183-a94c-91a6e1faddac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '254d381deace4007a67ecb9ee7b6144b'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:27.783703 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1240/2489] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:56:27 2022] DELETE /identity/v3/users/254d381deace4007a67ecb9ee7b6144b => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:56:28.040333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:56:28.041922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:56:28.055695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec1ec8-e5d0-453e-8467-c5817c9bbb5c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:56:28.118288 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba2294-f2e3-414e-805f-d8fbda21910a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:56:28.118941 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba2294-f2e3-414e-805f-d8fbda21910a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:56:28.119262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba2294-f2e3-414e-805f-d8fbda21910a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:56:28.119539 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba2294-f2e3-414e-805f-d8fbda21910a admin admin] PATH_INFO: `/v3/projects/e3a87f423aab4aeba8f4d971f878514c` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:56:28.130886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba2294-f2e3-414e-805f-d8fbda21910a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3a87f423aab4aeba8f4d971f878514c)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:56:28.132888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba2294-f2e3-414e-805f-d8fbda21910a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:56:28.230038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:56:28.239462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-22ba2294-f2e3-414e-805f-d8fbda21910a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3a87f423aab4aeba8f4d971f878514c', 'request_id': 'req-22ba2294-f2e3-414e-805f-d8fbda21910a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:28.361591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-22ba2294-f2e3-414e-805f-d8fbda21910a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3a87f423aab4aeba8f4d971f878514c', 'request_id': 'req-22ba2294-f2e3-414e-805f-d8fbda21910a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:28.369413 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-22ba2294-f2e3-414e-805f-d8fbda21910a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3a87f423aab4aeba8f4d971f878514c', 'request_id': 'req-22ba2294-f2e3-414e-805f-d8fbda21910a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:28.380638 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-22ba2294-f2e3-414e-805f-d8fbda21910a admin admin] The token cache is being invalidate because project e3a87f423aab4aeba8f4d971f878514c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:56:28.380928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-22ba2294-f2e3-414e-805f-d8fbda21910a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:28.381579 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-22ba2294-f2e3-414e-805f-d8fbda21910a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:56:28.383075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1250/2490] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:56:28 2022] DELETE /identity/v3/projects/e3a87f423aab4aeba8f4d971f878514c => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:56:30.924569 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689b2a4-38ce-4183-a94c-91a6e1faddac admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:56:30.926583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d689b2a4-38ce-4183-a94c-91a6e1faddac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:56:30.989388 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:56:31.008766 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689b2a4-38ce-4183-a94c-91a6e1faddac admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:56:31.085786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64519ed3-a0ca-436b-82ec-efd4c7d5fe4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:56:31.086409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64519ed3-a0ca-436b-82ec-efd4c7d5fe4f service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:56:31.086688 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64519ed3-a0ca-436b-82ec-efd4c7d5fe4f service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:56:31.086960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64519ed3-a0ca-436b-82ec-efd4c7d5fe4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:56:31.088772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-64519ed3-a0ca-436b-82ec-efd4c7d5fe4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:56:31.101211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64519ed3-a0ca-436b-82ec-efd4c7d5fe4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:56:31.102805 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64519ed3-a0ca-436b-82ec-efd4c7d5fe4f service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:56:31.118747 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1241/2491] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:56:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:56:38.444355 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:56:41.255669 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=35 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:16.284556 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba2294-f2e3-414e-805f-d8fbda21910a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:16.286150 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-22ba2294-f2e3-414e-805f-d8fbda21910a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:16.352907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba2294-f2e3-414e-805f-d8fbda21910a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:16.356545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a8b5e-6c14-4196-bdb5-c5ebe6ab0a00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:16.357147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a8b5e-6c14-4196-bdb5-c5ebe6ab0a00 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:16.357429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a8b5e-6c14-4196-bdb5-c5ebe6ab0a00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:16.357700 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a8b5e-6c14-4196-bdb5-c5ebe6ab0a00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:16.359481 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-587a8b5e-6c14-4196-bdb5-c5ebe6ab0a00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:16.448386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a8b5e-6c14-4196-bdb5-c5ebe6ab0a00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:16.449782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a8b5e-6c14-4196-bdb5-c5ebe6ab0a00 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:16.469010 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1251/2492] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:57:16.765993 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64519ed3-a0ca-436b-82ec-efd4c7d5fe4f service nova] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:16.768437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-64519ed3-a0ca-436b-82ec-efd4c7d5fe4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:16.831589 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64519ed3-a0ca-436b-82ec-efd4c7d5fe4f service nova] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:16.837344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ccd7f-9084-4cb5-8b61-9c4e721a231d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:16.838581 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ccd7f-9084-4cb5-8b61-9c4e721a231d service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:16.838581 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ccd7f-9084-4cb5-8b61-9c4e721a231d service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:16.838581 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ccd7f-9084-4cb5-8b61-9c4e721a231d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:16.840379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e64ccd7f-9084-4cb5-8b61-9c4e721a231d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:16.920750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ccd7f-9084-4cb5-8b61-9c4e721a231d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:16.922202 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ccd7f-9084-4cb5-8b61-9c4e721a231d service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:16.941796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1242/2493] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:20.232027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a8b5e-6c14-4196-bdb5-c5ebe6ab0a00 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:20.233525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-587a8b5e-6c14-4196-bdb5-c5ebe6ab0a00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:20.246546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a8b5e-6c14-4196-bdb5-c5ebe6ab0a00 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:20.251287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd24025-3e8a-4516-98f2-eb350d6535ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:20.251935 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd24025-3e8a-4516-98f2-eb350d6535ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.252216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd24025-3e8a-4516-98f2-eb350d6535ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.252489 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd24025-3e8a-4516-98f2-eb350d6535ab admin admin] PATH_INFO: `/v3/users/405b3189506a44bc93555c40917b4cbe` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.253619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd24025-3e8a-4516-98f2-eb350d6535ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=405b3189506a44bc93555c40917b4cbe)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:20.255569 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd24025-3e8a-4516-98f2-eb350d6535ab admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:20.472309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bfd24025-3e8a-4516-98f2-eb350d6535ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '405b3189506a44bc93555c40917b4cbe'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:20.484540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bfd24025-3e8a-4516-98f2-eb350d6535ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '405b3189506a44bc93555c40917b4cbe'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:20.491542 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bfd24025-3e8a-4516-98f2-eb350d6535ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '405b3189506a44bc93555c40917b4cbe'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:20.494094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:20.594649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bfd24025-3e8a-4516-98f2-eb350d6535ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '405b3189506a44bc93555c40917b4cbe'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:20.595830 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bfd24025-3e8a-4516-98f2-eb350d6535ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '405b3189506a44bc93555c40917b4cbe'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:20.602249 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1252/2494] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 11:57:20 2022] DELETE /identity/v3/users/405b3189506a44bc93555c40917b4cbe => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:20.859068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ccd7f-9084-4cb5-8b61-9c4e721a231d service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:20.872060 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ccd7f-9084-4cb5-8b61-9c4e721a231d service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:20.929601 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411277be-996a-443b-a4b5-9cc96ca56ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:20.930225 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411277be-996a-443b-a4b5-9cc96ca56ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:20.930497 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411277be-996a-443b-a4b5-9cc96ca56ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:20.930763 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411277be-996a-443b-a4b5-9cc96ca56ba7 admin admin] PATH_INFO: `/v3/projects/4767407e0fb04faaaa3a520128aa88df` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:20.931890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411277be-996a-443b-a4b5-9cc96ca56ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4767407e0fb04faaaa3a520128aa88df)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:20.933766 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411277be-996a-443b-a4b5-9cc96ca56ba7 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.064032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:21.070768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-411277be-996a-443b-a4b5-9cc96ca56ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4767407e0fb04faaaa3a520128aa88df', 'request_id': 'req-411277be-996a-443b-a4b5-9cc96ca56ba7'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:21.076316 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-411277be-996a-443b-a4b5-9cc96ca56ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4767407e0fb04faaaa3a520128aa88df', 'request_id': 'req-411277be-996a-443b-a4b5-9cc96ca56ba7'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:21.160633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-411277be-996a-443b-a4b5-9cc96ca56ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4767407e0fb04faaaa3a520128aa88df', 'request_id': 'req-411277be-996a-443b-a4b5-9cc96ca56ba7'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:21.172943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-411277be-996a-443b-a4b5-9cc96ca56ba7 admin admin] The token cache is being invalidate because project 4767407e0fb04faaaa3a520128aa88df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:57:21.173234 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-411277be-996a-443b-a4b5-9cc96ca56ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:21.173865 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-411277be-996a-443b-a4b5-9cc96ca56ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:57:21.175379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1243/2495] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 11:57:20 2022] DELETE /identity/v3/projects/4767407e0fb04faaaa3a520128aa88df => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:57:21.181981 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d7144-cc8c-45af-a7ef-0406136fb952 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.182279 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d7144-cc8c-45af-a7ef-0406136fb952 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.182561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d7144-cc8c-45af-a7ef-0406136fb952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.232674 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-606d7144-cc8c-45af-a7ef-0406136fb952 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:21.233905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-606d7144-cc8c-45af-a7ef-0406136fb952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:21.296202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1253/2496] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:21.304896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947cd7c-815c-4451-b170-bb47599a04ff None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.305172 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947cd7c-815c-4451-b170-bb47599a04ff None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.305442 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947cd7c-815c-4451-b170-bb47599a04ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.325956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-0947cd7c-815c-4451-b170-bb47599a04ff None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:21.327190 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-0947cd7c-815c-4451-b170-bb47599a04ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:21.335250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1244/2497] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:21.339786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d7144-cc8c-45af-a7ef-0406136fb952 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.349545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d7144-cc8c-45af-a7ef-0406136fb952 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.354166 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac9ab9-d44a-42d0-a199-951c03c9db17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.354767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac9ab9-d44a-42d0-a199-951c03c9db17 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.355047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac9ab9-d44a-42d0-a199-951c03c9db17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.355363 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac9ab9-d44a-42d0-a199-951c03c9db17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.356007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac9ab9-d44a-42d0-a199-951c03c9db17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:57:21.356311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac9ab9-d44a-42d0-a199-951c03c9db17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.357904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac9ab9-d44a-42d0-a199-951c03c9db17 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.370020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1254/2498] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 11:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:57:21.374322 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947cd7c-815c-4451-b170-bb47599a04ff None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.384065 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947cd7c-815c-4451-b170-bb47599a04ff None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.388902 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58360c66-74de-4e01-87ba-39579e82cd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.389504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58360c66-74de-4e01-87ba-39579e82cd4d admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.389783 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58360c66-74de-4e01-87ba-39579e82cd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.390055 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58360c66-74de-4e01-87ba-39579e82cd4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.390686 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58360c66-74de-4e01-87ba-39579e82cd4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.392768 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58360c66-74de-4e01-87ba-39579e82cd4d admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.523456 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1245/2499] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 11:57:21 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:57:21.527846 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac9ab9-d44a-42d0-a199-951c03c9db17 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.538830 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac9ab9-d44a-42d0-a199-951c03c9db17 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.595607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f7382-c1b0-4170-ba24-42db9588e230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.596280 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f7382-c1b0-4170-ba24-42db9588e230 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.596564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f7382-c1b0-4170-ba24-42db9588e230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.596834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f7382-c1b0-4170-ba24-42db9588e230 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.597487 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f7382-c1b0-4170-ba24-42db9588e230 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.599394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f7382-c1b0-4170-ba24-42db9588e230 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.723295 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1255/2500] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 11:57:21 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:57:21.727399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58360c66-74de-4e01-87ba-39579e82cd4d admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.737444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58360c66-74de-4e01-87ba-39579e82cd4d admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.741871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c5eac-5426-42ea-8473-7affe7583163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.742463 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c5eac-5426-42ea-8473-7affe7583163 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.742732 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c5eac-5426-42ea-8473-7affe7583163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.743000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c5eac-5426-42ea-8473-7affe7583163 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.743656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c5eac-5426-42ea-8473-7affe7583163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.745234 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c5eac-5426-42ea-8473-7affe7583163 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.755500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1246/2501] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:21.759618 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f7382-c1b0-4170-ba24-42db9588e230 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.770177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f7382-c1b0-4170-ba24-42db9588e230 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.774751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd18a3-6cbd-4a16-bbe8-ca3f17a14a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.781786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd18a3-6cbd-4a16-bbe8-ca3f17a14a88 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.781786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd18a3-6cbd-4a16-bbe8-ca3f17a14a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.781786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd18a3-6cbd-4a16-bbe8-ca3f17a14a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.781786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd18a3-6cbd-4a16-bbe8-ca3f17a14a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.781786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd18a3-6cbd-4a16-bbe8-ca3f17a14a88 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.788953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1256/2502] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 11:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:57:21.793146 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c5eac-5426-42ea-8473-7affe7583163 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:21.803555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c5eac-5426-42ea-8473-7affe7583163 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:21.808060 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686797ae-7c92-4018-84af-3abd3022413c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:21.808695 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686797ae-7c92-4018-84af-3abd3022413c admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.808971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686797ae-7c92-4018-84af-3abd3022413c admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.809246 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686797ae-7c92-4018-84af-3abd3022413c admin admin] PATH_INFO: `/v3/projects/edc640eda36b41d8b9cc59c635992d5b/users/7483b0e52765411d8444dfdd46549c5f/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.824139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686797ae-7c92-4018-84af-3abd3022413c admin admin] RBAC: Authorizing `identity:create_grant(project_id=edc640eda36b41d8b9cc59c635992d5b, user_id=7483b0e52765411d8444dfdd46549c5f, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:21.826757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686797ae-7c92-4018-84af-3abd3022413c admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:21.916455 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1247/2503] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 11:57:21 2022] PUT /identity/v3/projects/edc640eda36b41d8b9cc59c635992d5b/users/7483b0e52765411d8444dfdd46549c5f/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:57:21.922587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7b36b-bc96-4d5b-9cb6-307976ce1d61 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:21.923817 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7b36b-bc96-4d5b-9cb6-307976ce1d61 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:21.924347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7b36b-bc96-4d5b-9cb6-307976ce1d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:21.946942 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-1df7b36b-bc96-4d5b-9cb6-307976ce1d61 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:21.948207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1df7b36b-bc96-4d5b-9cb6-307976ce1d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:21.980844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:22.008714 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1257/2504] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:22.018539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686797ae-7c92-4018-84af-3abd3022413c admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:22.020269 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-686797ae-7c92-4018-84af-3abd3022413c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:22.077160 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686797ae-7c92-4018-84af-3abd3022413c admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:22.081042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51e75e-94e4-4c02-a1f3-f3034581d1df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:22.081702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51e75e-94e4-4c02-a1f3-f3034581d1df service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:22.081992 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51e75e-94e4-4c02-a1f3-f3034581d1df service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:22.082269 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51e75e-94e4-4c02-a1f3-f3034581d1df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:22.091779 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a51e75e-94e4-4c02-a1f3-f3034581d1df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:22.093207 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a51e75e-94e4-4c02-a1f3-f3034581d1df service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:22.102988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1248/2505] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:27.413768 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d643891-7f0a-4547-b455-a50280aa5b4a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:27.414382 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d643891-7f0a-4547-b455-a50280aa5b4a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:27.414786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d643891-7f0a-4547-b455-a50280aa5b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:27.447802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-8d643891-7f0a-4547-b455-a50280aa5b4a None None] MFA Rules not processed for user `7483b0e52765411d8444dfdd46549c5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:27.449040 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8d643891-7f0a-4547-b455-a50280aa5b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:27.517987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1258/2506] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:57:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:27.534435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51e75e-94e4-4c02-a1f3-f3034581d1df service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:27.535903 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5a51e75e-94e4-4c02-a1f3-f3034581d1df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:27.595541 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:27.614599 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51e75e-94e4-4c02-a1f3-f3034581d1df service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:27.620246 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5ba14-ac7a-4505-af73-37e47002b60e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:27.620864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5ba14-ac7a-4505-af73-37e47002b60e service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:27.621145 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5ba14-ac7a-4505-af73-37e47002b60e service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:27.621418 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5ba14-ac7a-4505-af73-37e47002b60e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:27.630276 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5ba14-ac7a-4505-af73-37e47002b60e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:27.631919 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5ba14-ac7a-4505-af73-37e47002b60e service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:27.642012 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1249/2507] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:30.283954 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4e2d1-124f-4611-9762-5a7d1ef0d50d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:30.284517 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4e2d1-124f-4611-9762-5a7d1ef0d50d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:30.284517 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4e2d1-124f-4611-9762-5a7d1ef0d50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:30.305549 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-41a4e2d1-124f-4611-9762-5a7d1ef0d50d None None] MFA Rules not processed for user `7483b0e52765411d8444dfdd46549c5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:57:30.312241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-41a4e2d1-124f-4611-9762-5a7d1ef0d50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:30.317308 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1259/2508] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 11:57:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:57:30.868733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5ba14-ac7a-4505-af73-37e47002b60e service nova] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:30.870474 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a6b5ba14-ac7a-4505-af73-37e47002b60e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:30.884039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5ba14-ac7a-4505-af73-37e47002b60e service nova] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:30.887767 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cf638-2f78-4b80-b7e1-4caed9554018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:30.891736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cf638-2f78-4b80-b7e1-4caed9554018 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:30.891736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cf638-2f78-4b80-b7e1-4caed9554018 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:30.891736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cf638-2f78-4b80-b7e1-4caed9554018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:30.891736 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-327cf638-2f78-4b80-b7e1-4caed9554018 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:30.901116 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cf638-2f78-4b80-b7e1-4caed9554018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:30.902554 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cf638-2f78-4b80-b7e1-4caed9554018 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:30.912312 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1250/2509] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:35.309909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4e2d1-124f-4611-9762-5a7d1ef0d50d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:57:35.323602 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4e2d1-124f-4611-9762-5a7d1ef0d50d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:57:35.329197 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf0d56-06f7-4c6a-ad07-1f8a6cb514ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:57:35.329802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf0d56-06f7-4c6a-ad07-1f8a6cb514ec service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:57:35.330081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf0d56-06f7-4c6a-ad07-1f8a6cb514ec service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:57:35.330367 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf0d56-06f7-4c6a-ad07-1f8a6cb514ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:57:35.332116 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-42cf0d56-06f7-4c6a-ad07-1f8a6cb514ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:57:35.344359 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf0d56-06f7-4c6a-ad07-1f8a6cb514ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:57:35.350014 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf0d56-06f7-4c6a-ad07-1f8a6cb514ec service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:57:35.357020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1260/2510] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:57:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:57:40.907927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=14 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:57:45.352314 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=51 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:16.572535 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cf638-2f78-4b80-b7e1-4caed9554018 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:16.573963 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-327cf638-2f78-4b80-b7e1-4caed9554018 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:16.632264 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cf638-2f78-4b80-b7e1-4caed9554018 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:16.636572 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f425fb-95d9-461b-9723-aaa79c74b3fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:16.637188 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f425fb-95d9-461b-9723-aaa79c74b3fe service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:16.637465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f425fb-95d9-461b-9723-aaa79c74b3fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:16.637744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f425fb-95d9-461b-9723-aaa79c74b3fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:16.639696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d3f425fb-95d9-461b-9723-aaa79c74b3fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:16.727184 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f425fb-95d9-461b-9723-aaa79c74b3fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:16.728539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f425fb-95d9-461b-9723-aaa79c74b3fe service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:16.750454 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1251/2511] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:16.870567 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf0d56-06f7-4c6a-ad07-1f8a6cb514ec service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:16.880444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf0d56-06f7-4c6a-ad07-1f8a6cb514ec service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:16.884104 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2b540-8c2b-4e3d-9273-1d9e2a250693 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:16.887479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2b540-8c2b-4e3d-9273-1d9e2a250693 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:16.887479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2b540-8c2b-4e3d-9273-1d9e2a250693 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:16.887479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2b540-8c2b-4e3d-9273-1d9e2a250693 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:16.887479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-11d2b540-8c2b-4e3d-9273-1d9e2a250693 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:16.899460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application [None req-11d2b540-8c2b-4e3d-9273-1d9e2a250693 service cinder] Could not find project: ec8ddabe0fd049de97da2f63453aabef.: keystone.exception.ProjectNotFound: Could not find project: ec8ddabe0fd049de97da2f63453aabef. Sep 19 11:58:16.899460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:58:16.899460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:58:16.899460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:58:16.899460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:58:16.899460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:58:16.899460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:58:16.899460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:58:16.899460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:58:16.899460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:58:16.899460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:58:16.899460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:58:16.899460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 19 11:58:16.899460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 19 11:58:16.899460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 19 11:58:16.900956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 19 11:58:16.900956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 19 11:58:16.900956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 19 11:58:16.900956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:58:16.900956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:58:16.900956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 19 11:58:16.900956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 19 11:58:16.900956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:58:16.900956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:58:16.900956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:58:16.900956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:58:16.900956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:58:16.900956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:58:16.900956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:58:16.900956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:58:16.900956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:58:16.900956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:58:16.902695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:58:16.902695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:58:16.902695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:58:16.902695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:58:16.902695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 19 11:58:16.902695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 19 11:58:16.902695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 19 11:58:16.902695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 19 11:58:16.902695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 19 11:58:16.902695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 19 11:58:16.902695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 19 11:58:16.902695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 19 11:58:16.902695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:58:16.902695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:58:16.902695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:58:16.902695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:58:16.902695 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:58:16.904196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:58:16.904196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:58:16.904196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application with Lock( Sep 19 11:58:16.904196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:58:16.904196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:58:16.904196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:58:16.904196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:58:16.904196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:58:16.904196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:58:16.904196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:58:16.904196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:58:16.904196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:58:16.904196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:58:16.904196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:58:16.904196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:58:16.904196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:58:16.904196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:58:16.906092 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: ec8ddabe0fd049de97da2f63453aabef. Sep 19 11:58:16.906092 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: ERROR keystone.server.flask.application Sep 19 11:58:16.906092 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1261/2512] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:16 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 31 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 19 11:58:17.587506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f425fb-95d9-461b-9723-aaa79c74b3fe service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:58:17.588966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d3f425fb-95d9-461b-9723-aaa79c74b3fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:17.656901 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f425fb-95d9-461b-9723-aaa79c74b3fe service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:58:17.662070 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a865a-17f7-4f1b-a2b9-81beafa4af13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:58:17.662951 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a865a-17f7-4f1b-a2b9-81beafa4af13 service placement] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:58:17.663260 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a865a-17f7-4f1b-a2b9-81beafa4af13 service placement] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:58:17.663536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a865a-17f7-4f1b-a2b9-81beafa4af13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:58:17.665260 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ac4a865a-17f7-4f1b-a2b9-81beafa4af13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:58:17.676748 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a865a-17f7-4f1b-a2b9-81beafa4af13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:58:17.678318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a865a-17f7-4f1b-a2b9-81beafa4af13 service placement] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:58:17.688149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1252/2513] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 11:58:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:58:26.890915 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:58:27.684264 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=93 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:00:16.492267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2b540-8c2b-4e3d-9273-1d9e2a250693 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:00:16.493627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-11d2b540-8c2b-4e3d-9273-1d9e2a250693 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:16.505516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2b540-8c2b-4e3d-9273-1d9e2a250693 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:00:16.509165 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e241a-fd4a-47cf-92e3-7b71dae7334b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:00:16.509761 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e241a-fd4a-47cf-92e3-7b71dae7334b service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:00:16.510045 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e241a-fd4a-47cf-92e3-7b71dae7334b service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:00:16.510320 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e241a-fd4a-47cf-92e3-7b71dae7334b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:00:16.512151 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-da6e241a-fd4a-47cf-92e3-7b71dae7334b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:00:16.583749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e241a-fd4a-47cf-92e3-7b71dae7334b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:00:16.585174 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e241a-fd4a-47cf-92e3-7b71dae7334b service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:00:16.594095 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1262/2514] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:00:16 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:00:26.589857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=33 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:08.509026 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a865a-17f7-4f1b-a2b9-81beafa4af13 service placement] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:08.518985 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a865a-17f7-4f1b-a2b9-81beafa4af13 service placement] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:08.522999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd12ad-daa1-42f6-8738-4a08ec8f2cc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:08.528423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd12ad-daa1-42f6-8738-4a08ec8f2cc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:08.528423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd12ad-daa1-42f6-8738-4a08ec8f2cc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:08.528423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd12ad-daa1-42f6-8738-4a08ec8f2cc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:08.528423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-0ccd12ad-daa1-42f6-8738-4a08ec8f2cc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:08.554042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd12ad-daa1-42f6-8738-4a08ec8f2cc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:08.555642 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd12ad-daa1-42f6-8738-4a08ec8f2cc2 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:08.567109 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1253/2515] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:01:12.451434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e241a-fd4a-47cf-92e3-7b71dae7334b service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:12.460652 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e241a-fd4a-47cf-92e3-7b71dae7334b service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:12.464253 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1780657-20a6-40e7-83d1-58f3c31ebd85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:12.464829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1780657-20a6-40e7-83d1-58f3c31ebd85 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:12.465114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1780657-20a6-40e7-83d1-58f3c31ebd85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:12.465388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1780657-20a6-40e7-83d1-58f3c31ebd85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:12.467115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e1780657-20a6-40e7-83d1-58f3c31ebd85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:12.481160 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1780657-20a6-40e7-83d1-58f3c31ebd85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:12.482606 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1780657-20a6-40e7-83d1-58f3c31ebd85 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:12.494635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1263/2516] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:01:17.430572 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd12ad-daa1-42f6-8738-4a08ec8f2cc2 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:17.442435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd12ad-daa1-42f6-8738-4a08ec8f2cc2 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:17.448379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a19c5-81c3-446c-8628-c4ec9897e838 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:17.448985 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a19c5-81c3-446c-8628-c4ec9897e838 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:17.449266 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a19c5-81c3-446c-8628-c4ec9897e838 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:17.449537 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a19c5-81c3-446c-8628-c4ec9897e838 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:17.451292 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4d2a19c5-81c3-446c-8628-c4ec9897e838 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:17.463806 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a19c5-81c3-446c-8628-c4ec9897e838 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:17.465382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a19c5-81c3-446c-8628-c4ec9897e838 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:17.477329 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1254/2517] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:01:18.678799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1780657-20a6-40e7-83d1-58f3c31ebd85 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:18.687551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1780657-20a6-40e7-83d1-58f3c31ebd85 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:18.691266 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ece15-d9a5-4892-9a24-3b2c7642783c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:18.691891 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ece15-d9a5-4892-9a24-3b2c7642783c service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:18.692159 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ece15-d9a5-4892-9a24-3b2c7642783c service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:18.692432 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ece15-d9a5-4892-9a24-3b2c7642783c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:18.694210 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4b2ece15-d9a5-4892-9a24-3b2c7642783c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:18.706457 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ece15-d9a5-4892-9a24-3b2c7642783c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:18.707946 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ece15-d9a5-4892-9a24-3b2c7642783c service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:18.718168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1264/2518] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:01:22.505315 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a19c5-81c3-446c-8628-c4ec9897e838 service nova] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:22.506904 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4d2a19c5-81c3-446c-8628-c4ec9897e838 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:22.526277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a19c5-81c3-446c-8628-c4ec9897e838 service nova] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:22.531141 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee3be7-52e0-4e03-a25b-c7bfd5d685ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:22.532439 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee3be7-52e0-4e03-a25b-c7bfd5d685ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:22.532439 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee3be7-52e0-4e03-a25b-c7bfd5d685ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:22.532439 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee3be7-52e0-4e03-a25b-c7bfd5d685ba admin admin] PATH_INFO: `/v3/users/d6fee905418f4d879da460b6758d4f23` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:22.533543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee3be7-52e0-4e03-a25b-c7bfd5d685ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6fee905418f4d879da460b6758d4f23)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:22.535505 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee3be7-52e0-4e03-a25b-c7bfd5d685ba admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:22.697414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-feee3be7-52e0-4e03-a25b-c7bfd5d685ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6fee905418f4d879da460b6758d4f23'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:22.710514 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-feee3be7-52e0-4e03-a25b-c7bfd5d685ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6fee905418f4d879da460b6758d4f23'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:22.718330 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-feee3be7-52e0-4e03-a25b-c7bfd5d685ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6fee905418f4d879da460b6758d4f23'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:22.783592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-feee3be7-52e0-4e03-a25b-c7bfd5d685ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6fee905418f4d879da460b6758d4f23'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:22.784523 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-feee3be7-52e0-4e03-a25b-c7bfd5d685ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6fee905418f4d879da460b6758d4f23'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:22.791881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1255/2519] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:01:22 2022] DELETE /identity/v3/users/d6fee905418f4d879da460b6758d4f23 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:01:23.105919 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ece15-d9a5-4892-9a24-3b2c7642783c service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:23.131963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ece15-d9a5-4892-9a24-3b2c7642783c service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:23.210536 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d476b-1779-4344-a4d8-e949ee7f02f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:23.211924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d476b-1779-4344-a4d8-e949ee7f02f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:23.211924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d476b-1779-4344-a4d8-e949ee7f02f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:23.211924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d476b-1779-4344-a4d8-e949ee7f02f2 admin admin] PATH_INFO: `/v3/projects/7f4d504e02a645b6a5ebc78ed0bc5a41` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:23.217395 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d476b-1779-4344-a4d8-e949ee7f02f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f4d504e02a645b6a5ebc78ed0bc5a41)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:23.217395 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d476b-1779-4344-a4d8-e949ee7f02f2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:23.351106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-080d476b-1779-4344-a4d8-e949ee7f02f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f4d504e02a645b6a5ebc78ed0bc5a41', 'request_id': 'req-080d476b-1779-4344-a4d8-e949ee7f02f2'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:23.417697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-080d476b-1779-4344-a4d8-e949ee7f02f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f4d504e02a645b6a5ebc78ed0bc5a41', 'request_id': 'req-080d476b-1779-4344-a4d8-e949ee7f02f2'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:23.425472 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-080d476b-1779-4344-a4d8-e949ee7f02f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f4d504e02a645b6a5ebc78ed0bc5a41', 'request_id': 'req-080d476b-1779-4344-a4d8-e949ee7f02f2'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:23.436149 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-080d476b-1779-4344-a4d8-e949ee7f02f2 admin admin] The token cache is being invalidate because project 7f4d504e02a645b6a5ebc78ed0bc5a41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:01:23.436440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-080d476b-1779-4344-a4d8-e949ee7f02f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:23.437083 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-080d476b-1779-4344-a4d8-e949ee7f02f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:23.438561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1265/2520] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:01:23 2022] DELETE /identity/v3/projects/7f4d504e02a645b6a5ebc78ed0bc5a41 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:01:23.451155 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a31873-7927-447b-b0f3-0246300e6708 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:23.451657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a31873-7927-447b-b0f3-0246300e6708 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:23.452178 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a31873-7927-447b-b0f3-0246300e6708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:23.487815 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-47a31873-7927-447b-b0f3-0246300e6708 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:01:23.489076 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-47a31873-7927-447b-b0f3-0246300e6708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:23.565755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1256/2521] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:01:23.568193 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcfd95-ccbc-4563-a2cf-53fd1fce5369 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:23.572824 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcfd95-ccbc-4563-a2cf-53fd1fce5369 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:23.572824 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcfd95-ccbc-4563-a2cf-53fd1fce5369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:23.589454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-acfcfd95-ccbc-4563-a2cf-53fd1fce5369 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:01:23.590772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-acfcfd95-ccbc-4563-a2cf-53fd1fce5369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:23.600393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1266/2522] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:01:23.604812 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a31873-7927-447b-b0f3-0246300e6708 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:23.614542 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a31873-7927-447b-b0f3-0246300e6708 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:23.621942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a3d4b-befc-4d78-8fad-dc9e06f7743c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:23.622544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a3d4b-befc-4d78-8fad-dc9e06f7743c admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:23.622823 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a3d4b-befc-4d78-8fad-dc9e06f7743c admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:23.623098 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a3d4b-befc-4d78-8fad-dc9e06f7743c admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:23.623801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a3d4b-befc-4d78-8fad-dc9e06f7743c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:01:23.624128 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a3d4b-befc-4d78-8fad-dc9e06f7743c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:23.625686 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a3d4b-befc-4d78-8fad-dc9e06f7743c admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:23.637781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1257/2523] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:01:23.642151 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfcfd95-ccbc-4563-a2cf-53fd1fce5369 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:23.652094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfcfd95-ccbc-4563-a2cf-53fd1fce5369 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:23.657991 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1984f8-22b9-43bf-813c-33afb122b68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:23.658656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1984f8-22b9-43bf-813c-33afb122b68b admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:23.658977 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1984f8-22b9-43bf-813c-33afb122b68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:23.660372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1984f8-22b9-43bf-813c-33afb122b68b admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:23.660990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1984f8-22b9-43bf-813c-33afb122b68b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:23.662897 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1984f8-22b9-43bf-813c-33afb122b68b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:23.757825 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1267/2524] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:01:23 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:01:23.759009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a3d4b-befc-4d78-8fad-dc9e06f7743c admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:23.770101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a3d4b-befc-4d78-8fad-dc9e06f7743c admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:23.779352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:23.826882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c83771-26d1-44dc-8563-2da935913458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:23.827442 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c83771-26d1-44dc-8563-2da935913458 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:23.827724 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c83771-26d1-44dc-8563-2da935913458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:23.827995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c83771-26d1-44dc-8563-2da935913458 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:23.828627 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c83771-26d1-44dc-8563-2da935913458 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:23.830623 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c83771-26d1-44dc-8563-2da935913458 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:23.935157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1258/2525] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:01:23 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:01:23.939426 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1984f8-22b9-43bf-813c-33afb122b68b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:23.952784 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1984f8-22b9-43bf-813c-33afb122b68b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:23.958293 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05888b-3566-4433-a3c4-8d375fe72ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:23.958914 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05888b-3566-4433-a3c4-8d375fe72ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:23.959276 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05888b-3566-4433-a3c4-8d375fe72ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:23.959498 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05888b-3566-4433-a3c4-8d375fe72ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:23.962386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05888b-3566-4433-a3c4-8d375fe72ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:23.962386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05888b-3566-4433-a3c4-8d375fe72ff7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:23.972766 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1268/2526] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:01:23.976799 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c83771-26d1-44dc-8563-2da935913458 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:23.987128 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c83771-26d1-44dc-8563-2da935913458 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:23.993005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d630366-1ca1-4259-a38f-e05dbe24d121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:23.993005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d630366-1ca1-4259-a38f-e05dbe24d121 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:23.993005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d630366-1ca1-4259-a38f-e05dbe24d121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:23.993426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d630366-1ca1-4259-a38f-e05dbe24d121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:23.994064 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d630366-1ca1-4259-a38f-e05dbe24d121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:23.995698 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d630366-1ca1-4259-a38f-e05dbe24d121 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:24.007162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1259/2527] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:01:24.011887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05888b-3566-4433-a3c4-8d375fe72ff7 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:24.026060 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05888b-3566-4433-a3c4-8d375fe72ff7 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:24.031004 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b42f4e-a16b-4817-ac69-e3965430d613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:24.031752 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b42f4e-a16b-4817-ac69-e3965430d613 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:24.032029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b42f4e-a16b-4817-ac69-e3965430d613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:24.032306 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b42f4e-a16b-4817-ac69-e3965430d613 admin admin] PATH_INFO: `/v3/projects/a62ca9fb60f047468bc92d4e1fef95d0/users/047eee2591d14b828bbc74c8ce57cfa5/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:24.059932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b42f4e-a16b-4817-ac69-e3965430d613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a62ca9fb60f047468bc92d4e1fef95d0, user_id=047eee2591d14b828bbc74c8ce57cfa5, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:24.059932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b42f4e-a16b-4817-ac69-e3965430d613 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:24.122234 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:24.127324 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1269/2528] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:01:24 2022] PUT /identity/v3/projects/a62ca9fb60f047468bc92d4e1fef95d0/users/047eee2591d14b828bbc74c8ce57cfa5/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:01:24.196554 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c96da-0f4f-47e6-ba9a-a0ea7426f322 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:24.196813 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c96da-0f4f-47e6-ba9a-a0ea7426f322 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:24.197064 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c96da-0f4f-47e6-ba9a-a0ea7426f322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:24.232232 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-801c96da-0f4f-47e6-ba9a-a0ea7426f322 None None] MFA Rules not processed for user `047eee2591d14b828bbc74c8ce57cfa5`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:01:24.233740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-801c96da-0f4f-47e6-ba9a-a0ea7426f322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:24.298753 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1260/2529] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:01:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:01:24.303497 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b42f4e-a16b-4817-ac69-e3965430d613 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:24.374000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b42f4e-a16b-4817-ac69-e3965430d613 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:24.378885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c97f4-6a82-4b5d-9015-c0585f3f441a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:24.379534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c97f4-6a82-4b5d-9015-c0585f3f441a admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:24.383855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c97f4-6a82-4b5d-9015-c0585f3f441a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:24.383855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c97f4-6a82-4b5d-9015-c0585f3f441a admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:24.383855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c97f4-6a82-4b5d-9015-c0585f3f441a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:24.383855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c97f4-6a82-4b5d-9015-c0585f3f441a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:24.499754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1270/2530] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:01:24 2022] POST /identity/v3/projects => generated 379 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:01:24.500960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c96da-0f4f-47e6-ba9a-a0ea7426f322 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:24.513251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c96da-0f4f-47e6-ba9a-a0ea7426f322 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:24.571677 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2928760-b343-46a2-92ac-8817acb96695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:24.572593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2928760-b343-46a2-92ac-8817acb96695 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:24.572593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2928760-b343-46a2-92ac-8817acb96695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:24.572929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2928760-b343-46a2-92ac-8817acb96695 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:24.573458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2928760-b343-46a2-92ac-8817acb96695 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:24.575398 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2928760-b343-46a2-92ac-8817acb96695 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:24.677022 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1261/2531] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:01:24 2022] POST /identity/v3/users => generated 296 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:01:24.680972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c97f4-6a82-4b5d-9015-c0585f3f441a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:24.691618 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c97f4-6a82-4b5d-9015-c0585f3f441a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:24.696353 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6c4c1-58c0-493e-a03a-d0c519ebfc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:24.696980 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6c4c1-58c0-493e-a03a-d0c519ebfc7d admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:24.697265 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6c4c1-58c0-493e-a03a-d0c519ebfc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:24.697543 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6c4c1-58c0-493e-a03a-d0c519ebfc7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:24.698197 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6c4c1-58c0-493e-a03a-d0c519ebfc7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:24.699909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6c4c1-58c0-493e-a03a-d0c519ebfc7d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:24.714495 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1271/2532] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:01:24.718994 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2928760-b343-46a2-92ac-8817acb96695 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:24.732917 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2928760-b343-46a2-92ac-8817acb96695 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:24.739263 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53452e-f2ad-4526-893c-2ed1c6f69822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:24.739928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53452e-f2ad-4526-893c-2ed1c6f69822 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:24.740394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53452e-f2ad-4526-893c-2ed1c6f69822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:24.740925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53452e-f2ad-4526-893c-2ed1c6f69822 admin admin] PATH_INFO: `/v3/projects/f3810440fe3042c1b8b3978fcfff53bd/users/9eecaf981cf2432496c04eb32b1c1d68/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:24.750148 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:24.757073 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53452e-f2ad-4526-893c-2ed1c6f69822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3810440fe3042c1b8b3978fcfff53bd, user_id=9eecaf981cf2432496c04eb32b1c1d68, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:24.759757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53452e-f2ad-4526-893c-2ed1c6f69822 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:24.861691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1262/2533] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:01:24 2022] PUT /identity/v3/projects/f3810440fe3042c1b8b3978fcfff53bd/users/9eecaf981cf2432496c04eb32b1c1d68/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:01:24.866532 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6c4c1-58c0-493e-a03a-d0c519ebfc7d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:24.877079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6c4c1-58c0-493e-a03a-d0c519ebfc7d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:24.932906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a8b3b-3e87-4d5b-9fa2-0cc510bc23ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:24.933511 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a8b3b-3e87-4d5b-9fa2-0cc510bc23ab admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:24.933791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a8b3b-3e87-4d5b-9fa2-0cc510bc23ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:24.934070 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a8b3b-3e87-4d5b-9fa2-0cc510bc23ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:24.934694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a8b3b-3e87-4d5b-9fa2-0cc510bc23ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:24.936335 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a8b3b-3e87-4d5b-9fa2-0cc510bc23ab admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:24.946718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1272/2534] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:01:24.950806 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53452e-f2ad-4526-893c-2ed1c6f69822 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:24.962311 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53452e-f2ad-4526-893c-2ed1c6f69822 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:24.967643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd39dce-a9f5-4288-924a-72fbbe5e9539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:24.969012 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd39dce-a9f5-4288-924a-72fbbe5e9539 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:24.969277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd39dce-a9f5-4288-924a-72fbbe5e9539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:24.969622 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd39dce-a9f5-4288-924a-72fbbe5e9539 admin admin] PATH_INFO: `/v3/projects/f3810440fe3042c1b8b3978fcfff53bd/users/9eecaf981cf2432496c04eb32b1c1d68/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:24.972767 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd39dce-a9f5-4288-924a-72fbbe5e9539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3810440fe3042c1b8b3978fcfff53bd, user_id=9eecaf981cf2432496c04eb32b1c1d68, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:24.975352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd39dce-a9f5-4288-924a-72fbbe5e9539 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:24.991263 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1263/2535] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:01:24 2022] PUT /identity/v3/projects/f3810440fe3042c1b8b3978fcfff53bd/users/9eecaf981cf2432496c04eb32b1c1d68/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:01:25.004691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ba8d3-de55-45a3-b2d1-561f7f4f1d34 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:25.004975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ba8d3-de55-45a3-b2d1-561f7f4f1d34 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:25.005267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ba8d3-de55-45a3-b2d1-561f7f4f1d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:25.037395 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-b82ba8d3-de55-45a3-b2d1-561f7f4f1d34 None None] MFA Rules not processed for user `9eecaf981cf2432496c04eb32b1c1d68`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:01:25.038898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b82ba8d3-de55-45a3-b2d1-561f7f4f1d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:25.109038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:25.120204 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1273/2536] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:01:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:01:25.132165 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd39dce-a9f5-4288-924a-72fbbe5e9539 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:25.137122 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-cdd39dce-a9f5-4288-924a-72fbbe5e9539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:25.205942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd39dce-a9f5-4288-924a-72fbbe5e9539 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:25.209931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20535d57-cae1-4635-b6ba-235ab7e69401 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:25.210544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20535d57-cae1-4635-b6ba-235ab7e69401 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:25.210823 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20535d57-cae1-4635-b6ba-235ab7e69401 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:25.211118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20535d57-cae1-4635-b6ba-235ab7e69401 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:25.222065 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20535d57-cae1-4635-b6ba-235ab7e69401 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:25.224340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20535d57-cae1-4635-b6ba-235ab7e69401 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:25.236808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1264/2537] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:01:25.496577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ba8d3-de55-45a3-b2d1-561f7f4f1d34 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:25.506373 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ba8d3-de55-45a3-b2d1-561f7f4f1d34 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:25.509966 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864c445-21e4-4ec6-84a3-985b6098393a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:25.510568 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864c445-21e4-4ec6-84a3-985b6098393a service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:25.510848 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864c445-21e4-4ec6-84a3-985b6098393a service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:25.511116 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864c445-21e4-4ec6-84a3-985b6098393a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:25.575241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0864c445-21e4-4ec6-84a3-985b6098393a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:25.576800 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0864c445-21e4-4ec6-84a3-985b6098393a service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:25.586855 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1274/2538] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:01:26.761372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89d325-011f-477a-a1c2-098261cf8190 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:26.761924 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89d325-011f-477a-a1c2-098261cf8190 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:26.761924 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89d325-011f-477a-a1c2-098261cf8190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:26.781531 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-7b89d325-011f-477a-a1c2-098261cf8190 None None] MFA Rules not processed for user `7483b0e52765411d8444dfdd46549c5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:01:26.782654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7b89d325-011f-477a-a1c2-098261cf8190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:26.841331 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1265/2539] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:01:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:01:27.227985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864c445-21e4-4ec6-84a3-985b6098393a service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:27.229557 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0864c445-21e4-4ec6-84a3-985b6098393a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:27.309226 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864c445-21e4-4ec6-84a3-985b6098393a service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:27.313981 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c656b-b4f7-40b6-bfa6-e9b1cb1bb161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:27.314586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c656b-b4f7-40b6-bfa6-e9b1cb1bb161 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:27.314864 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c656b-b4f7-40b6-bfa6-e9b1cb1bb161 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:27.320977 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c656b-b4f7-40b6-bfa6-e9b1cb1bb161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:27.320977 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5f4c656b-b4f7-40b6-bfa6-e9b1cb1bb161 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:27.327935 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c656b-b4f7-40b6-bfa6-e9b1cb1bb161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:27.329358 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c656b-b4f7-40b6-bfa6-e9b1cb1bb161 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:27.337966 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1275/2540] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:01:28.766254 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89d325-011f-477a-a1c2-098261cf8190 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:28.767668 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7b89d325-011f-477a-a1c2-098261cf8190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:28.838656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89d325-011f-477a-a1c2-098261cf8190 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:28.844546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bae01-cefb-4a3b-98cb-e017aece4129 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:28.845160 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bae01-cefb-4a3b-98cb-e017aece4129 service placement] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:28.845663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bae01-cefb-4a3b-98cb-e017aece4129 service placement] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:28.845944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bae01-cefb-4a3b-98cb-e017aece4129 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:28.848608 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-fa4bae01-cefb-4a3b-98cb-e017aece4129 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:28.861336 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bae01-cefb-4a3b-98cb-e017aece4129 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:28.863209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bae01-cefb-4a3b-98cb-e017aece4129 service placement] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:28.877269 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1266/2541] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:01:33.185777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ed25d-d35c-4d8e-92e0-138912f7fa23 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:33.185777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ed25d-d35c-4d8e-92e0-138912f7fa23 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:33.185777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ed25d-d35c-4d8e-92e0-138912f7fa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:33.212224 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a16ed25d-d35c-4d8e-92e0-138912f7fa23 None None] MFA Rules not processed for user `5a667fa07b1148998959835d9995ad60`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:01:33.213441 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a16ed25d-d35c-4d8e-92e0-138912f7fa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:33.283210 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1276/2542] 10.222.0.78 () {58 vars in 1009 bytes} [Mon Sep 19 12:01:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:01:33.434006 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bae01-cefb-4a3b-98cb-e017aece4129 service placement] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:33.435568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-fa4bae01-cefb-4a3b-98cb-e017aece4129 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:33.450194 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:33.507744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bae01-cefb-4a3b-98cb-e017aece4129 service placement] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:33.512882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646132d-6537-4fad-a4b5-2f9a1e3942ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf462b2aa3fc4e2ca58851549cf0c644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:33.513704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646132d-6537-4fad-a4b5-2f9a1e3942ac service swift] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:33.513977 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646132d-6537-4fad-a4b5-2f9a1e3942ac service swift] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:33.514248 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646132d-6537-4fad-a4b5-2f9a1e3942ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:33.524217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646132d-6537-4fad-a4b5-2f9a1e3942ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:33.525850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646132d-6537-4fad-a4b5-2f9a1e3942ac service swift] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:33.535590 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1267/2543] 10.222.0.78 () {60 vars in 1384 bytes} [Mon Sep 19 12:01:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 12:01:34.930304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ed25d-d35c-4d8e-92e0-138912f7fa23 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:34.931782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a16ed25d-d35c-4d8e-92e0-138912f7fa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:34.970330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:35.100490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ed25d-d35c-4d8e-92e0-138912f7fa23 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:35.110400 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3120a2-172d-4b8b-8895-0c83323aeb13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:35.110400 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3120a2-172d-4b8b-8895-0c83323aeb13 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:35.110400 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3120a2-172d-4b8b-8895-0c83323aeb13 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:35.110400 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3120a2-172d-4b8b-8895-0c83323aeb13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:35.118604 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fb3120a2-172d-4b8b-8895-0c83323aeb13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:35.140912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3120a2-172d-4b8b-8895-0c83323aeb13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:35.142707 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3120a2-172d-4b8b-8895-0c83323aeb13 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:35.157955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1277/2544] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:01:35.798688 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646132d-6537-4fad-a4b5-2f9a1e3942ac service swift] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:35.800521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7646132d-6537-4fad-a4b5-2f9a1e3942ac service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:35.816991 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646132d-6537-4fad-a4b5-2f9a1e3942ac service swift] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:35.822196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05258c-b73b-453d-87fa-952e4cb46c5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:35.822819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05258c-b73b-453d-87fa-952e4cb46c5e service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:35.823098 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05258c-b73b-453d-87fa-952e4cb46c5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:35.823445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05258c-b73b-453d-87fa-952e4cb46c5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:35.825621 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ab05258c-b73b-453d-87fa-952e4cb46c5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:35.840010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05258c-b73b-453d-87fa-952e4cb46c5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:35.841527 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05258c-b73b-453d-87fa-952e4cb46c5e service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:35.852540 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1268/2545] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:01:38.251971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3120a2-172d-4b8b-8895-0c83323aeb13 service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:38.362570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3120a2-172d-4b8b-8895-0c83323aeb13 service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:38.368624 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd999502-9821-4f11-a178-d42664ee0a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:38.369553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd999502-9821-4f11-a178-d42664ee0a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:38.369921 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd999502-9821-4f11-a178-d42664ee0a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:38.370291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd999502-9821-4f11-a178-d42664ee0a0e admin admin] PATH_INFO: `/v3/users/047eee2591d14b828bbc74c8ce57cfa5` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:38.372467 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd999502-9821-4f11-a178-d42664ee0a0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=047eee2591d14b828bbc74c8ce57cfa5)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:38.378064 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd999502-9821-4f11-a178-d42664ee0a0e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:38.607847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bd999502-9821-4f11-a178-d42664ee0a0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '047eee2591d14b828bbc74c8ce57cfa5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:38.624984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bd999502-9821-4f11-a178-d42664ee0a0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '047eee2591d14b828bbc74c8ce57cfa5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:38.642563 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bd999502-9821-4f11-a178-d42664ee0a0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '047eee2591d14b828bbc74c8ce57cfa5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:38.737505 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bd999502-9821-4f11-a178-d42664ee0a0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '047eee2591d14b828bbc74c8ce57cfa5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:38.738657 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-bd999502-9821-4f11-a178-d42664ee0a0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '047eee2591d14b828bbc74c8ce57cfa5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:38.750514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1278/2546] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:01:38 2022] DELETE /identity/v3/users/047eee2591d14b828bbc74c8ce57cfa5 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:01:38.755117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dbec6-1d27-4add-ad02-285268accdcc None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:38.755953 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dbec6-1d27-4add-ad02-285268accdcc None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:38.756122 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dbec6-1d27-4add-ad02-285268accdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:38.782176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-673dbec6-1d27-4add-ad02-285268accdcc None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:01:38.783825 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-673dbec6-1d27-4add-ad02-285268accdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:38.862814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1269/2547] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:01:38.873192 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd999502-9821-4f11-a178-d42664ee0a0e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:38.876302 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bd999502-9821-4f11-a178-d42664ee0a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:38.921066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:38.978875 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd999502-9821-4f11-a178-d42664ee0a0e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:38.987110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce9d3f-653e-461d-b74b-54a67277b21c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:38.987903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce9d3f-653e-461d-b74b-54a67277b21c service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:38.988188 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce9d3f-653e-461d-b74b-54a67277b21c service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:38.988464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce9d3f-653e-461d-b74b-54a67277b21c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:38.998854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce9d3f-653e-461d-b74b-54a67277b21c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:39.000339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce9d3f-653e-461d-b74b-54a67277b21c service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:39.015773 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1279/2548] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:01:39.519682 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dbec6-1d27-4add-ad02-285268accdcc None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:39.521625 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-673dbec6-1d27-4add-ad02-285268accdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:39.551258 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dbec6-1d27-4add-ad02-285268accdcc None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:39.562516 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:39.563209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:39.563487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:39.563767 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c admin admin] PATH_INFO: `/v3/projects/a62ca9fb60f047468bc92d4e1fef95d0` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:39.568009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a62ca9fb60f047468bc92d4e1fef95d0)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:39.571362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:39.706655 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a62ca9fb60f047468bc92d4e1fef95d0', 'request_id': 'req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:39.720809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a62ca9fb60f047468bc92d4e1fef95d0', 'request_id': 'req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:39.814081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a62ca9fb60f047468bc92d4e1fef95d0', 'request_id': 'req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:39.831877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c admin admin] The token cache is being invalidate because project a62ca9fb60f047468bc92d4e1fef95d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:01:39.832445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:39.833358 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:39.835596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1270/2549] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:01:39 2022] DELETE /identity/v3/projects/a62ca9fb60f047468bc92d4e1fef95d0 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:01:39.839910 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce9d3f-653e-461d-b74b-54a67277b21c service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:39.851979 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce9d3f-653e-461d-b74b-54a67277b21c service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:39.937293 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72646415-2633-424e-9c67-c4875c7f8a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:39.938835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72646415-2633-424e-9c67-c4875c7f8a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:39.938835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72646415-2633-424e-9c67-c4875c7f8a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:39.938835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72646415-2633-424e-9c67-c4875c7f8a2a admin admin] PATH_INFO: `/v3/users/9eecaf981cf2432496c04eb32b1c1d68` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:39.940017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72646415-2633-424e-9c67-c4875c7f8a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eecaf981cf2432496c04eb32b1c1d68)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:39.941994 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72646415-2633-424e-9c67-c4875c7f8a2a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:40.006066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:40.007928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:40.025489 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:40.087812 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf3f11-d56f-476c-bca8-0e7a2ff0380c admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:40.091765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5b860-479a-40f5-8284-c351067718c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:40.095312 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5b860-479a-40f5-8284-c351067718c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:40.095312 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5b860-479a-40f5-8284-c351067718c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:40.095312 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5b860-479a-40f5-8284-c351067718c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:40.095312 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-94c5b860-479a-40f5-8284-c351067718c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:40.116215 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5b860-479a-40f5-8284-c351067718c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:40.118971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5b860-479a-40f5-8284-c351067718c6 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:40.130279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1271/2550] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:01:40.178164 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-72646415-2633-424e-9c67-c4875c7f8a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eecaf981cf2432496c04eb32b1c1d68'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:40.199340 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-72646415-2633-424e-9c67-c4875c7f8a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eecaf981cf2432496c04eb32b1c1d68'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:40.207496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-72646415-2633-424e-9c67-c4875c7f8a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eecaf981cf2432496c04eb32b1c1d68'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:40.315344 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-72646415-2633-424e-9c67-c4875c7f8a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eecaf981cf2432496c04eb32b1c1d68'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:40.316168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-72646415-2633-424e-9c67-c4875c7f8a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eecaf981cf2432496c04eb32b1c1d68'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:40.327678 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1280/2551] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:01:39 2022] DELETE /identity/v3/users/9eecaf981cf2432496c04eb32b1c1d68 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:01:40.926075 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5b860-479a-40f5-8284-c351067718c6 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:40.927809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-94c5b860-479a-40f5-8284-c351067718c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:41.042164 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5b860-479a-40f5-8284-c351067718c6 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:41.128459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7944e4-4dd3-4ce8-af54-86f4d8d0c8a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:41.129069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7944e4-4dd3-4ce8-af54-86f4d8d0c8a2 service placement] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:41.129351 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7944e4-4dd3-4ce8-af54-86f4d8d0c8a2 service placement] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:41.129624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7944e4-4dd3-4ce8-af54-86f4d8d0c8a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:41.131438 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-af7944e4-4dd3-4ce8-af54-86f4d8d0c8a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:41.142480 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7944e4-4dd3-4ce8-af54-86f4d8d0c8a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:41.144106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7944e4-4dd3-4ce8-af54-86f4d8d0c8a2 service placement] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:41.154081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1272/2552] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:01:41.448637 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72646415-2633-424e-9c67-c4875c7f8a2a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:41.451947 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-72646415-2633-424e-9c67-c4875c7f8a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:41.501273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7944e4-4dd3-4ce8-af54-86f4d8d0c8a2 service placement] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:41.502985 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-af7944e4-4dd3-4ce8-af54-86f4d8d0c8a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:41.590104 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:41.611847 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:41.618802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72646415-2633-424e-9c67-c4875c7f8a2a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:41.633152 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7944e4-4dd3-4ce8-af54-86f4d8d0c8a2 service placement] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:41.636164 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f15e46-a18d-4238-97e9-1c7c7d4a1993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:41.636868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f15e46-a18d-4238-97e9-1c7c7d4a1993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:41.637154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f15e46-a18d-4238-97e9-1c7c7d4a1993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:41.637434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f15e46-a18d-4238-97e9-1c7c7d4a1993 admin admin] PATH_INFO: `/v3/projects/f3810440fe3042c1b8b3978fcfff53bd` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:41.638664 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f15e46-a18d-4238-97e9-1c7c7d4a1993 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3810440fe3042c1b8b3978fcfff53bd)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:41.640238 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbb66d-594a-45bf-a096-2612ac6a95f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:41.640709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f15e46-a18d-4238-97e9-1c7c7d4a1993 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:41.641267 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbb66d-594a-45bf-a096-2612ac6a95f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:41.641708 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbb66d-594a-45bf-a096-2612ac6a95f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:41.642167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbb66d-594a-45bf-a096-2612ac6a95f4 admin admin] PATH_INFO: `/v3/users/4682f63b56d24450b2605b5b4c2129d9` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:41.643670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbb66d-594a-45bf-a096-2612ac6a95f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4682f63b56d24450b2605b5b4c2129d9)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:41.645686 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbb66d-594a-45bf-a096-2612ac6a95f4 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:41.816296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-53bbb66d-594a-45bf-a096-2612ac6a95f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4682f63b56d24450b2605b5b4c2129d9'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:41.830212 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-27f15e46-a18d-4238-97e9-1c7c7d4a1993 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3810440fe3042c1b8b3978fcfff53bd', 'request_id': 'req-27f15e46-a18d-4238-97e9-1c7c7d4a1993'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:41.840018 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-53bbb66d-594a-45bf-a096-2612ac6a95f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4682f63b56d24450b2605b5b4c2129d9'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:41.860989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-53bbb66d-594a-45bf-a096-2612ac6a95f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4682f63b56d24450b2605b5b4c2129d9'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:41.894946 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-27f15e46-a18d-4238-97e9-1c7c7d4a1993 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3810440fe3042c1b8b3978fcfff53bd', 'request_id': 'req-27f15e46-a18d-4238-97e9-1c7c7d4a1993'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:41.906555 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-27f15e46-a18d-4238-97e9-1c7c7d4a1993 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3810440fe3042c1b8b3978fcfff53bd', 'request_id': 'req-27f15e46-a18d-4238-97e9-1c7c7d4a1993'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:41.920918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-27f15e46-a18d-4238-97e9-1c7c7d4a1993 admin admin] The token cache is being invalidate because project f3810440fe3042c1b8b3978fcfff53bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:01:41.921309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-27f15e46-a18d-4238-97e9-1c7c7d4a1993 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:41.922514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-27f15e46-a18d-4238-97e9-1c7c7d4a1993 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:41.928096 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1281/2553] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:01:41 2022] DELETE /identity/v3/projects/f3810440fe3042c1b8b3978fcfff53bd => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:01:41.930456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d917d-b509-47a3-8476-889b8cf9f8a1 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:41.931443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d917d-b509-47a3-8476-889b8cf9f8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:41.931813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d917d-b509-47a3-8476-889b8cf9f8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:41.958684 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-493d917d-b509-47a3-8476-889b8cf9f8a1 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:01:41.960150 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-493d917d-b509-47a3-8476-889b8cf9f8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:41.977934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-53bbb66d-594a-45bf-a096-2612ac6a95f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4682f63b56d24450b2605b5b4c2129d9'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:41.979679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-53bbb66d-594a-45bf-a096-2612ac6a95f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4682f63b56d24450b2605b5b4c2129d9'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:41.990195 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1273/2554] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:01:41 2022] DELETE /identity/v3/users/4682f63b56d24450b2605b5b4c2129d9 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:01:42.047286 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1282/2555] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:01:42.055303 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5834e-c906-4264-abda-c652f4826eb1 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:42.056051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5834e-c906-4264-abda-c652f4826eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:42.056711 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5834e-c906-4264-abda-c652f4826eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:42.086144 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-53e5834e-c906-4264-abda-c652f4826eb1 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:01:42.091505 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-53e5834e-c906-4264-abda-c652f4826eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:42.218294 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1274/2556] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:01:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:01:42.227114 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d917d-b509-47a3-8476-889b8cf9f8a1 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:42.256834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d917d-b509-47a3-8476-889b8cf9f8a1 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:42.272065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee02b31-0ee9-4745-9611-3d3acfce6cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:42.273286 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee02b31-0ee9-4745-9611-3d3acfce6cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:42.273564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee02b31-0ee9-4745-9611-3d3acfce6cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:42.273834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee02b31-0ee9-4745-9611-3d3acfce6cd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:42.274495 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee02b31-0ee9-4745-9611-3d3acfce6cd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:01:42.274794 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee02b31-0ee9-4745-9611-3d3acfce6cd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:42.276565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee02b31-0ee9-4745-9611-3d3acfce6cd5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:42.301051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1283/2557] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:01:42.306365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5834e-c906-4264-abda-c652f4826eb1 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:42.317878 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5834e-c906-4264-abda-c652f4826eb1 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:42.329235 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39ffc2-29d2-494c-ba5f-cbac249b9dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:42.330130 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39ffc2-29d2-494c-ba5f-cbac249b9dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:42.330520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39ffc2-29d2-494c-ba5f-cbac249b9dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:42.330896 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39ffc2-29d2-494c-ba5f-cbac249b9dd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:42.331803 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39ffc2-29d2-494c-ba5f-cbac249b9dd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:42.333982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39ffc2-29d2-494c-ba5f-cbac249b9dd7 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:42.456914 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee02b31-0ee9-4745-9611-3d3acfce6cd5 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:42.464924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fee02b31-0ee9-4745-9611-3d3acfce6cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:42.471587 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1275/2558] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:01:42 2022] POST /identity/v3/projects => generated 357 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:01:42.477683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39ffc2-29d2-494c-ba5f-cbac249b9dd7 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:42.480341 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee02b31-0ee9-4745-9611-3d3acfce6cd5 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:42.598474 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:42.610387 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d658c-f7bb-4a7c-9e17-37c3be216126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:42.611782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d658c-f7bb-4a7c-9e17-37c3be216126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:42.612455 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d658c-f7bb-4a7c-9e17-37c3be216126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:42.613886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d658c-f7bb-4a7c-9e17-37c3be216126 admin admin] PATH_INFO: `/v3/projects/575748e2f3ee4df18ce5aa403175b66f` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:42.617043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d658c-f7bb-4a7c-9e17-37c3be216126 admin admin] RBAC: Authorizing `identity:delete_project(project_id=575748e2f3ee4df18ce5aa403175b66f)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:42.621907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d658c-f7bb-4a7c-9e17-37c3be216126 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:42.633167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39ffc2-29d2-494c-ba5f-cbac249b9dd7 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:42.638488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2622c-b8dd-4539-8870-4eb1682a7ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:42.639487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2622c-b8dd-4539-8870-4eb1682a7ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:42.639879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2622c-b8dd-4539-8870-4eb1682a7ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:42.640261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2622c-b8dd-4539-8870-4eb1682a7ab4 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:42.640997 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2622c-b8dd-4539-8870-4eb1682a7ab4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:42.643092 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2622c-b8dd-4539-8870-4eb1682a7ab4 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:42.754034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1276/2559] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:01:42 2022] POST /identity/v3/users => generated 279 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:01:42.754034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2622c-b8dd-4539-8870-4eb1682a7ab4 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:42.772997 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2622c-b8dd-4539-8870-4eb1682a7ab4 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:42.778408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-160d658c-f7bb-4a7c-9e17-37c3be216126 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '575748e2f3ee4df18ce5aa403175b66f', 'request_id': 'req-160d658c-f7bb-4a7c-9e17-37c3be216126'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:42.845104 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf933830-507e-4380-bf81-24f9695c5f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:42.845742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf933830-507e-4380-bf81-24f9695c5f32 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:42.846098 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf933830-507e-4380-bf81-24f9695c5f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:42.846570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf933830-507e-4380-bf81-24f9695c5f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:42.847318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf933830-507e-4380-bf81-24f9695c5f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:42.851032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf933830-507e-4380-bf81-24f9695c5f32 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:42.863835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1277/2560] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:01:42.865669 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf933830-507e-4380-bf81-24f9695c5f32 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:42.869063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-160d658c-f7bb-4a7c-9e17-37c3be216126 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '575748e2f3ee4df18ce5aa403175b66f', 'request_id': 'req-160d658c-f7bb-4a7c-9e17-37c3be216126'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:42.877739 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-160d658c-f7bb-4a7c-9e17-37c3be216126 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '575748e2f3ee4df18ce5aa403175b66f', 'request_id': 'req-160d658c-f7bb-4a7c-9e17-37c3be216126'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:42.888301 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf933830-507e-4380-bf81-24f9695c5f32 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:42.898082 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab62296-6e8a-4a85-b520-1afeff47cd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:42.898082 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab62296-6e8a-4a85-b520-1afeff47cd94 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:42.898082 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab62296-6e8a-4a85-b520-1afeff47cd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:42.898473 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab62296-6e8a-4a85-b520-1afeff47cd94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:42.899015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab62296-6e8a-4a85-b520-1afeff47cd94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:42.899553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:42.901301 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab62296-6e8a-4a85-b520-1afeff47cd94 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:42.905452 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-160d658c-f7bb-4a7c-9e17-37c3be216126 admin admin] The token cache is being invalidate because project 575748e2f3ee4df18ce5aa403175b66f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:01:42.905888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-160d658c-f7bb-4a7c-9e17-37c3be216126 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:42.907484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-160d658c-f7bb-4a7c-9e17-37c3be216126 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:01:42.909494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1284/2561] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:01:42 2022] DELETE /identity/v3/projects/575748e2f3ee4df18ce5aa403175b66f => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:01:42.914854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89817c00-d961-4280-b69c-d91ef0c84cd5 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:42.915296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89817c00-d961-4280-b69c-d91ef0c84cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:42.915716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89817c00-d961-4280-b69c-d91ef0c84cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:42.917750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1278/2562] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:01:42.922052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab62296-6e8a-4a85-b520-1afeff47cd94 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:42.925123 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5ab62296-6e8a-4a85-b520-1afeff47cd94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:42.951976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab62296-6e8a-4a85-b520-1afeff47cd94 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:42.953366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-89817c00-d961-4280-b69c-d91ef0c84cd5 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:01:42.954924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-89817c00-d961-4280-b69c-d91ef0c84cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:42.972395 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1285/2563] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:01:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:01:43.190265 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6cae3-33f9-4247-bed1-23c209a2e934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:43.190265 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6cae3-33f9-4247-bed1-23c209a2e934 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:43.190890 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dc7c2-d0ca-4b72-b6cb-917ee5f1d117 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:43.190890 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dc7c2-d0ca-4b72-b6cb-917ee5f1d117 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:43.190890 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dc7c2-d0ca-4b72-b6cb-917ee5f1d117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:43.191080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6cae3-33f9-4247-bed1-23c209a2e934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:43.191080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6cae3-33f9-4247-bed1-23c209a2e934 admin admin] PATH_INFO: `/v3/projects/9e7cdb81067d4e1dbdb3feecbc785504/users/404e7327415647b480fb2788f178ea64/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:43.191080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6cae3-33f9-4247-bed1-23c209a2e934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e7cdb81067d4e1dbdb3feecbc785504, user_id=404e7327415647b480fb2788f178ea64, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:43.191689 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ee5dc7c2-d0ca-4b72-b6cb-917ee5f1d117 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:01:43.191765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6cae3-33f9-4247-bed1-23c209a2e934 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:43.191834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ee5dc7c2-d0ca-4b72-b6cb-917ee5f1d117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:43.191834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1286/2564] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:01:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:01:43.191834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5dc7c2-d0ca-4b72-b6cb-917ee5f1d117 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:43.191834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5dc7c2-d0ca-4b72-b6cb-917ee5f1d117 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:43.192070 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d8a8c-b56d-476c-90d6-cadc64600d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:43.192070 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d8a8c-b56d-476c-90d6-cadc64600d79 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:43.192070 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d8a8c-b56d-476c-90d6-cadc64600d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:43.192070 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d8a8c-b56d-476c-90d6-cadc64600d79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:43.192311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d8a8c-b56d-476c-90d6-cadc64600d79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:01:43.192311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d8a8c-b56d-476c-90d6-cadc64600d79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:43.192311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d8a8c-b56d-476c-90d6-cadc64600d79 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:43.192311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1287/2565] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:01:43.192311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d8a8c-b56d-476c-90d6-cadc64600d79 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:43.192311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d8a8c-b56d-476c-90d6-cadc64600d79 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:43.192669 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45329fba-7ad7-47a0-81f3-cae2b5ef1877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:43.192669 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45329fba-7ad7-47a0-81f3-cae2b5ef1877 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:43.192669 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45329fba-7ad7-47a0-81f3-cae2b5ef1877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:43.192669 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45329fba-7ad7-47a0-81f3-cae2b5ef1877 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:43.192929 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45329fba-7ad7-47a0-81f3-cae2b5ef1877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:43.192929 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45329fba-7ad7-47a0-81f3-cae2b5ef1877 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:43.193047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1279/2566] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:01:42 2022] PUT /identity/v3/projects/9e7cdb81067d4e1dbdb3feecbc785504/users/404e7327415647b480fb2788f178ea64/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:01:43.193047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139b0e5-c372-4e03-9dd8-44a24e2a536d None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:43.193047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139b0e5-c372-4e03-9dd8-44a24e2a536d None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:43.193047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139b0e5-c372-4e03-9dd8-44a24e2a536d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:43.193047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-2139b0e5-c372-4e03-9dd8-44a24e2a536d None None] MFA Rules not processed for user `404e7327415647b480fb2788f178ea64`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:01:43.193422 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2139b0e5-c372-4e03-9dd8-44a24e2a536d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:43.289947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1280/2567] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:01:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:01:43.295808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139b0e5-c372-4e03-9dd8-44a24e2a536d None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:43.384469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139b0e5-c372-4e03-9dd8-44a24e2a536d None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:43.388939 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1288/2568] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:01:43 2022] POST /identity/v3/projects => generated 391 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:01:43.393056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ccf2b2-e22a-4931-9d6a-04a3c7378c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:43.393202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45329fba-7ad7-47a0-81f3-cae2b5ef1877 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:43.393692 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccf2b2-e22a-4931-9d6a-04a3c7378c2a admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:43.393965 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccf2b2-e22a-4931-9d6a-04a3c7378c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:43.394248 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccf2b2-e22a-4931-9d6a-04a3c7378c2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:43.394876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ccf2b2-e22a-4931-9d6a-04a3c7378c2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:43.397355 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ccf2b2-e22a-4931-9d6a-04a3c7378c2a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:43.492077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1281/2569] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:01:43 2022] POST /identity/v3/projects => generated 355 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:01:43.494265 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ccf2b2-e22a-4931-9d6a-04a3c7378c2a admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:43.506033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45329fba-7ad7-47a0-81f3-cae2b5ef1877 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:43.634450 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ccf2b2-e22a-4931-9d6a-04a3c7378c2a admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:43.638059 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:43.642363 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13fac3-6c75-4d80-9971-f6ce476a1e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:43.642962 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13fac3-6c75-4d80-9971-f6ce476a1e2b admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:43.643338 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13fac3-6c75-4d80-9971-f6ce476a1e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:43.643618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13fac3-6c75-4d80-9971-f6ce476a1e2b admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:43.643710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7c628-c0bc-43b8-b1d3-628da6f6b2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:43.644302 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13fac3-6c75-4d80-9971-f6ce476a1e2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:43.644668 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7c628-c0bc-43b8-b1d3-628da6f6b2ce admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:43.646992 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13fac3-6c75-4d80-9971-f6ce476a1e2b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:43.647249 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7c628-c0bc-43b8-b1d3-628da6f6b2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:43.647698 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7c628-c0bc-43b8-b1d3-628da6f6b2ce admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:43.648535 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7c628-c0bc-43b8-b1d3-628da6f6b2ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:43.652015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7c628-c0bc-43b8-b1d3-628da6f6b2ce admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:43.771945 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:43.774175 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1282/2571] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:01:43 2022] POST /identity/v3/users => generated 284 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:01:43.774360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1289/2571] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:01:43 2022] POST /identity/v3/users => generated 296 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:01:43.776197 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7c628-c0bc-43b8-b1d3-628da6f6b2ce admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:43.781507 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13fac3-6c75-4d80-9971-f6ce476a1e2b admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:43.801652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13fac3-6c75-4d80-9971-f6ce476a1e2b admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:43.801939 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7c628-c0bc-43b8-b1d3-628da6f6b2ce admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:43.824608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76520920-818d-47c4-b71c-6dd0e00bfc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:43.824608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76520920-818d-47c4-b71c-6dd0e00bfc5c admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:43.824608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76520920-818d-47c4-b71c-6dd0e00bfc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:43.824608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76520920-818d-47c4-b71c-6dd0e00bfc5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:43.825112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2aaa15-f68f-455d-a847-176b0db04dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:43.825193 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76520920-818d-47c4-b71c-6dd0e00bfc5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:43.825256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aaa15-f68f-455d-a847-176b0db04dff admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:43.825256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aaa15-f68f-455d-a847-176b0db04dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:43.825256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aaa15-f68f-455d-a847-176b0db04dff admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:43.825256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2aaa15-f68f-455d-a847-176b0db04dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:43.825459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76520920-818d-47c4-b71c-6dd0e00bfc5c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:43.827628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2aaa15-f68f-455d-a847-176b0db04dff admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:43.836694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1290/2572] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:01:43.843188 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76520920-818d-47c4-b71c-6dd0e00bfc5c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:43.844999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1283/2573] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:01:43.846366 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2aaa15-f68f-455d-a847-176b0db04dff admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:43.871256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2aaa15-f68f-455d-a847-176b0db04dff admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:43.871451 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76520920-818d-47c4-b71c-6dd0e00bfc5c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:43.876976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec566da-5d2f-4c37-ae9b-3cf805e0a49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:43.877687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec566da-5d2f-4c37-ae9b-3cf805e0a49b admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:43.878092 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec566da-5d2f-4c37-ae9b-3cf805e0a49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:43.878470 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec566da-5d2f-4c37-ae9b-3cf805e0a49b admin admin] PATH_INFO: `/v3/projects/38bda36e8f2c438fa1cb56228e16db1f/users/be44a831be4b4f25a778495b19b1e800/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:43.885657 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85c7c7-6d4f-4854-8c21-36834ed28ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:43.886286 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85c7c7-6d4f-4854-8c21-36834ed28ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:43.887003 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85c7c7-6d4f-4854-8c21-36834ed28ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:43.887003 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85c7c7-6d4f-4854-8c21-36834ed28ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:43.887598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85c7c7-6d4f-4854-8c21-36834ed28ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:43.890419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85c7c7-6d4f-4854-8c21-36834ed28ba2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:43.913744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec566da-5d2f-4c37-ae9b-3cf805e0a49b admin admin] RBAC: Authorizing `identity:create_grant(project_id=38bda36e8f2c438fa1cb56228e16db1f, user_id=be44a831be4b4f25a778495b19b1e800, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:43.917757 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1291/2574] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:01:43.918141 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec566da-5d2f-4c37-ae9b-3cf805e0a49b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:43.918967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85c7c7-6d4f-4854-8c21-36834ed28ba2 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:43.932502 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85c7c7-6d4f-4854-8c21-36834ed28ba2 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:43.940530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437bf27-404c-43c7-b1ca-34ae8b627ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:43.941100 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437bf27-404c-43c7-b1ca-34ae8b627ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:43.941383 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437bf27-404c-43c7-b1ca-34ae8b627ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:43.941657 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437bf27-404c-43c7-b1ca-34ae8b627ba3 admin admin] PATH_INFO: `/v3/projects/f1244521558b4d218afb0744e9fd9bd2/users/aedbcefd8c484ddfac8039956c2cec70/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:43.961210 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437bf27-404c-43c7-b1ca-34ae8b627ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1244521558b4d218afb0744e9fd9bd2, user_id=aedbcefd8c484ddfac8039956c2cec70, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:43.963943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437bf27-404c-43c7-b1ca-34ae8b627ba3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:43.993243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1284/2575] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:01:43 2022] PUT /identity/v3/projects/38bda36e8f2c438fa1cb56228e16db1f/users/be44a831be4b4f25a778495b19b1e800/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:01:43.994328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec566da-5d2f-4c37-ae9b-3cf805e0a49b admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:44.081410 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1292/2576] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:01:43 2022] PUT /identity/v3/projects/f1244521558b4d218afb0744e9fd9bd2/users/aedbcefd8c484ddfac8039956c2cec70/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:01:44.083039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec566da-5d2f-4c37-ae9b-3cf805e0a49b admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:44.093833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c7c55-ee29-4428-9350-5d1f473ccc65 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:44.094170 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c7c55-ee29-4428-9350-5d1f473ccc65 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:44.094261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:44.094490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c7c55-ee29-4428-9350-5d1f473ccc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:44.143836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-c35c7c55-ee29-4428-9350-5d1f473ccc65 None None] MFA Rules not processed for user `aedbcefd8c484ddfac8039956c2cec70`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:01:44.145609 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c35c7c55-ee29-4428-9350-5d1f473ccc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:44.178978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64113d-8527-41fd-8ee2-abde119dd60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:44.179916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64113d-8527-41fd-8ee2-abde119dd60c admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:44.180322 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64113d-8527-41fd-8ee2-abde119dd60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:44.181316 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64113d-8527-41fd-8ee2-abde119dd60c admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:44.182106 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64113d-8527-41fd-8ee2-abde119dd60c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:44.184697 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64113d-8527-41fd-8ee2-abde119dd60c admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:44.203009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1285/2577] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:01:44.205025 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64113d-8527-41fd-8ee2-abde119dd60c admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:44.218765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64113d-8527-41fd-8ee2-abde119dd60c admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:44.228491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4f742-dff1-444c-b732-fac1aabb9772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:44.229423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4f742-dff1-444c-b732-fac1aabb9772 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:44.229959 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4f742-dff1-444c-b732-fac1aabb9772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:44.230244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4f742-dff1-444c-b732-fac1aabb9772 admin admin] PATH_INFO: `/v3/projects/38bda36e8f2c438fa1cb56228e16db1f/users/be44a831be4b4f25a778495b19b1e800/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:44.232376 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4f742-dff1-444c-b732-fac1aabb9772 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38bda36e8f2c438fa1cb56228e16db1f, user_id=be44a831be4b4f25a778495b19b1e800, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:44.235662 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4f742-dff1-444c-b732-fac1aabb9772 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:44.252102 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1286/2578] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:01:44 2022] PUT /identity/v3/projects/38bda36e8f2c438fa1cb56228e16db1f/users/be44a831be4b4f25a778495b19b1e800/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:01:44.261936 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdd67e-918a-4f31-9c10-89c36bb924d6 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:44.262372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdd67e-918a-4f31-9c10-89c36bb924d6 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:44.263297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdd67e-918a-4f31-9c10-89c36bb924d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:44.267208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1293/2579] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:01:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:01:44.278744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c7c55-ee29-4428-9350-5d1f473ccc65 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:44.280468 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c35c7c55-ee29-4428-9350-5d1f473ccc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:44.297296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-dbcdd67e-918a-4f31-9c10-89c36bb924d6 None None] MFA Rules not processed for user `be44a831be4b4f25a778495b19b1e800`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:01:44.298966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-dbcdd67e-918a-4f31-9c10-89c36bb924d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:44.361394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c7c55-ee29-4428-9350-5d1f473ccc65 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:44.403659 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1287/2580] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:01:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:01:44.411650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcdd67e-918a-4f31-9c10-89c36bb924d6 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:44.415040 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-dbcdd67e-918a-4f31-9c10-89c36bb924d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:44.436795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:44.445013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9dbc4-4fd4-4962-bc5e-1e3bba148f35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:44.445963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9dbc4-4fd4-4962-bc5e-1e3bba148f35 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:44.447240 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9dbc4-4fd4-4962-bc5e-1e3bba148f35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:44.447240 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9dbc4-4fd4-4962-bc5e-1e3bba148f35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:44.500385 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcdd67e-918a-4f31-9c10-89c36bb924d6 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:44.505293 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d01fe4-8a15-439b-bbc7-2cbe88bb4642 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:44.506109 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d01fe4-8a15-439b-bbc7-2cbe88bb4642 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:44.506583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d01fe4-8a15-439b-bbc7-2cbe88bb4642 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:44.507224 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d01fe4-8a15-439b-bbc7-2cbe88bb4642 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:44.526163 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d01fe4-8a15-439b-bbc7-2cbe88bb4642 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:44.527888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9dbc4-4fd4-4962-bc5e-1e3bba148f35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:44.528352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d01fe4-8a15-439b-bbc7-2cbe88bb4642 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:44.529345 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9dbc4-4fd4-4962-bc5e-1e3bba148f35 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:44.540087 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1288/2581] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:01:44.542412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1294/2582] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:01:44.824382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d01fe4-8a15-439b-bbc7-2cbe88bb4642 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:44.834373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d01fe4-8a15-439b-bbc7-2cbe88bb4642 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:44.840162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d87fd-f33a-4d74-a79f-f8858f4dd54e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:44.842719 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d87fd-f33a-4d74-a79f-f8858f4dd54e service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:44.842719 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d87fd-f33a-4d74-a79f-f8858f4dd54e service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:44.842719 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d87fd-f33a-4d74-a79f-f8858f4dd54e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:44.878462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:44.945809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d87fd-f33a-4d74-a79f-f8858f4dd54e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:44.945809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d87fd-f33a-4d74-a79f-f8858f4dd54e service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:44.956218 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1289/2583] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:44 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:01:54.535148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=28 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:55.491947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=18 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:01:55.785892 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9dbc4-4fd4-4962-bc5e-1e3bba148f35 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:55.787399 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1ba9dbc4-4fd4-4962-bc5e-1e3bba148f35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:55.876530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9dbc4-4fd4-4962-bc5e-1e3bba148f35 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:55.880162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16a107-5b17-4ab9-b5bf-2100aed191f6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558e3b6dd9754c57ad028aa431a68b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:55.880770 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16a107-5b17-4ab9-b5bf-2100aed191f6 service glance] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:55.881053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16a107-5b17-4ab9-b5bf-2100aed191f6 service glance] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:55.881336 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16a107-5b17-4ab9-b5bf-2100aed191f6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:55.883090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bc16a107-5b17-4ab9-b5bf-2100aed191f6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:55.953870 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16a107-5b17-4ab9-b5bf-2100aed191f6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:55.955584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16a107-5b17-4ab9-b5bf-2100aed191f6 service glance] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:55.967579 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1295/2584] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:01:56.176932 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d87fd-f33a-4d74-a79f-f8858f4dd54e service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:56.179133 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-164d87fd-f33a-4d74-a79f-f8858f4dd54e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:56.237159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d87fd-f33a-4d74-a79f-f8858f4dd54e service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:56.241965 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54bcd1f-41db-46a6-8204-9a158c6cec98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:56.242575 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bcd1f-41db-46a6-8204-9a158c6cec98 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:56.242851 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bcd1f-41db-46a6-8204-9a158c6cec98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:56.243121 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bcd1f-41db-46a6-8204-9a158c6cec98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:56.245355 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-f54bcd1f-41db-46a6-8204-9a158c6cec98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:56.257472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54bcd1f-41db-46a6-8204-9a158c6cec98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:56.258916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54bcd1f-41db-46a6-8204-9a158c6cec98 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:56.269746 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1290/2585] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:01:56.365669 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16a107-5b17-4ab9-b5bf-2100aed191f6 service glance] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:01:56.367655 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-bc16a107-5b17-4ab9-b5bf-2100aed191f6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:56.382433 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16a107-5b17-4ab9-b5bf-2100aed191f6 service glance] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:01:56.388552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4dc29-903a-48be-8ec3-fb4d0f6c2090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:01:56.389469 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4dc29-903a-48be-8ec3-fb4d0f6c2090 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:01:56.389882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4dc29-903a-48be-8ec3-fb4d0f6c2090 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:01:56.390297 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4dc29-903a-48be-8ec3-fb4d0f6c2090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:01:56.394369 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e1b4dc29-903a-48be-8ec3-fb4d0f6c2090 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:01:56.413780 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4dc29-903a-48be-8ec3-fb4d0f6c2090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:01:56.415261 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4dc29-903a-48be-8ec3-fb4d0f6c2090 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:01:56.430391 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1296/2586] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:01:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:02:01.330029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27407cc-e1cb-4d82-92cb-d11312c13b80 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:01.330493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27407cc-e1cb-4d82-92cb-d11312c13b80 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:01.330631 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27407cc-e1cb-4d82-92cb-d11312c13b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:01.351598 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-e27407cc-e1cb-4d82-92cb-d11312c13b80 None None] MFA Rules not processed for user `5a667fa07b1148998959835d9995ad60`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:01.352816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e27407cc-e1cb-4d82-92cb-d11312c13b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:01.409590 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1291/2587] 10.222.0.78 () {58 vars in 1009 bytes} [Mon Sep 19 12:02:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:01.421592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4dc29-903a-48be-8ec3-fb4d0f6c2090 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:01.423030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e1b4dc29-903a-48be-8ec3-fb4d0f6c2090 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:01.501411 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4dc29-903a-48be-8ec3-fb4d0f6c2090 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:01.506654 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c381a-ceb8-4bc3-ba4b-ebce330caf6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf462b2aa3fc4e2ca58851549cf0c644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:01.507801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c381a-ceb8-4bc3-ba4b-ebce330caf6c service swift] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:01.508092 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c381a-ceb8-4bc3-ba4b-ebce330caf6c service swift] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:01.508363 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c381a-ceb8-4bc3-ba4b-ebce330caf6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:01.516692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c381a-ceb8-4bc3-ba4b-ebce330caf6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:01.518276 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c381a-ceb8-4bc3-ba4b-ebce330caf6c service swift] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:01.522118 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:01.528365 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1297/2588] 10.222.0.78 () {60 vars in 1384 bytes} [Mon Sep 19 12:02:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 12:02:06.298485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27407cc-e1cb-4d82-92cb-d11312c13b80 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:06.383894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27407cc-e1cb-4d82-92cb-d11312c13b80 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:06.390240 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816f333-b535-4a25-98f6-85fa2e4a6727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:06.391798 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816f333-b535-4a25-98f6-85fa2e4a6727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:06.392535 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816f333-b535-4a25-98f6-85fa2e4a6727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:06.393206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816f333-b535-4a25-98f6-85fa2e4a6727 admin admin] PATH_INFO: `/v3/users/aedbcefd8c484ddfac8039956c2cec70` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:06.395907 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816f333-b535-4a25-98f6-85fa2e4a6727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aedbcefd8c484ddfac8039956c2cec70)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:06.400028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816f333-b535-4a25-98f6-85fa2e4a6727 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:06.584184 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e816f333-b535-4a25-98f6-85fa2e4a6727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aedbcefd8c484ddfac8039956c2cec70'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:06.586644 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:06.598847 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e816f333-b535-4a25-98f6-85fa2e4a6727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aedbcefd8c484ddfac8039956c2cec70'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:06.606165 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e816f333-b535-4a25-98f6-85fa2e4a6727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aedbcefd8c484ddfac8039956c2cec70'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:06.737264 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e816f333-b535-4a25-98f6-85fa2e4a6727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aedbcefd8c484ddfac8039956c2cec70'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:06.766882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e816f333-b535-4a25-98f6-85fa2e4a6727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aedbcefd8c484ddfac8039956c2cec70'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:06.766882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1292/2589] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:02:06 2022] DELETE /identity/v3/users/aedbcefd8c484ddfac8039956c2cec70 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:06.767098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e804817-5582-4ec7-b198-4de3d8ca7a78 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:06.767098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e804817-5582-4ec7-b198-4de3d8ca7a78 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:06.767098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e804817-5582-4ec7-b198-4de3d8ca7a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:06.796427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-4e804817-5582-4ec7-b198-4de3d8ca7a78 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:06.797950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4e804817-5582-4ec7-b198-4de3d8ca7a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:06.873070 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1298/2590] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:06.889712 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816f333-b535-4a25-98f6-85fa2e4a6727 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:06.889712 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e816f333-b535-4a25-98f6-85fa2e4a6727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:06.988152 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816f333-b535-4a25-98f6-85fa2e4a6727 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:06.992531 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86c4ee-902d-406e-92d9-47ea0c9e273a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:06.993324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86c4ee-902d-406e-92d9-47ea0c9e273a service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:06.993603 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86c4ee-902d-406e-92d9-47ea0c9e273a service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:06.993880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86c4ee-902d-406e-92d9-47ea0c9e273a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:07.015719 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86c4ee-902d-406e-92d9-47ea0c9e273a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:07.017169 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86c4ee-902d-406e-92d9-47ea0c9e273a service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:07.028256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1293/2591] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:02:07.570537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e804817-5582-4ec7-b198-4de3d8ca7a78 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:07.572041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4e804817-5582-4ec7-b198-4de3d8ca7a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:07.592360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e804817-5582-4ec7-b198-4de3d8ca7a78 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:07.615570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c8e60-633f-430e-942d-38247c63fe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:07.616389 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c8e60-633f-430e-942d-38247c63fe39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:07.616568 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c8e60-633f-430e-942d-38247c63fe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:07.616882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c8e60-633f-430e-942d-38247c63fe39 admin admin] PATH_INFO: `/v3/projects/f1244521558b4d218afb0744e9fd9bd2` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:07.619051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c8e60-633f-430e-942d-38247c63fe39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1244521558b4d218afb0744e9fd9bd2)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:07.621037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c8e60-633f-430e-942d-38247c63fe39 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:07.783442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-407c8e60-633f-430e-942d-38247c63fe39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1244521558b4d218afb0744e9fd9bd2', 'request_id': 'req-407c8e60-633f-430e-942d-38247c63fe39'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:07.858334 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-407c8e60-633f-430e-942d-38247c63fe39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1244521558b4d218afb0744e9fd9bd2', 'request_id': 'req-407c8e60-633f-430e-942d-38247c63fe39'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:07.866497 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-407c8e60-633f-430e-942d-38247c63fe39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1244521558b4d218afb0744e9fd9bd2', 'request_id': 'req-407c8e60-633f-430e-942d-38247c63fe39'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:07.879632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-407c8e60-633f-430e-942d-38247c63fe39 admin admin] The token cache is being invalidate because project f1244521558b4d218afb0744e9fd9bd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:02:07.879827 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-407c8e60-633f-430e-942d-38247c63fe39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:07.880508 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-407c8e60-633f-430e-942d-38247c63fe39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:07.899221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1299/2592] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:02:07 2022] DELETE /identity/v3/projects/f1244521558b4d218afb0744e9fd9bd2 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:07.930493 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86c4ee-902d-406e-92d9-47ea0c9e273a service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:07.932750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-8c86c4ee-902d-406e-92d9-47ea0c9e273a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:07.947332 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86c4ee-902d-406e-92d9-47ea0c9e273a service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:08.045469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92446d-6b74-4c57-9534-6fbc98afe38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:08.046428 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92446d-6b74-4c57-9534-6fbc98afe38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:08.046772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92446d-6b74-4c57-9534-6fbc98afe38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:08.047145 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92446d-6b74-4c57-9534-6fbc98afe38f admin admin] PATH_INFO: `/v3/users/404e7327415647b480fb2788f178ea64` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:08.048785 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92446d-6b74-4c57-9534-6fbc98afe38f admin admin] RBAC: Authorizing `identity:delete_user(user_id=404e7327415647b480fb2788f178ea64)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:08.051610 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92446d-6b74-4c57-9534-6fbc98afe38f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:08.251299 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-df92446d-6b74-4c57-9534-6fbc98afe38f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '404e7327415647b480fb2788f178ea64'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:08.265478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-df92446d-6b74-4c57-9534-6fbc98afe38f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '404e7327415647b480fb2788f178ea64'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:08.274784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:08.277345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-df92446d-6b74-4c57-9534-6fbc98afe38f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '404e7327415647b480fb2788f178ea64'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:08.351091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-df92446d-6b74-4c57-9534-6fbc98afe38f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '404e7327415647b480fb2788f178ea64'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:08.351771 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-df92446d-6b74-4c57-9534-6fbc98afe38f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '404e7327415647b480fb2788f178ea64'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:08.362320 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1294/2593] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:02:07 2022] DELETE /identity/v3/users/404e7327415647b480fb2788f178ea64 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:08.367787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f4e27-2ee2-428c-95b8-2198082e84a6 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:08.368083 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f4e27-2ee2-428c-95b8-2198082e84a6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:08.368369 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f4e27-2ee2-428c-95b8-2198082e84a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:08.398093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-731f4e27-2ee2-428c-95b8-2198082e84a6 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:08.399354 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-731f4e27-2ee2-428c-95b8-2198082e84a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:08.469143 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1300/2594] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:08.479809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92446d-6b74-4c57-9534-6fbc98afe38f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:08.481247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-df92446d-6b74-4c57-9534-6fbc98afe38f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:08.547704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92446d-6b74-4c57-9534-6fbc98afe38f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:08.551620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5e0a3-a0d7-4231-95d1-8624255983c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:08.552258 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5e0a3-a0d7-4231-95d1-8624255983c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:08.552505 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5e0a3-a0d7-4231-95d1-8624255983c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:08.552817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5e0a3-a0d7-4231-95d1-8624255983c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:08.564204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5e0a3-a0d7-4231-95d1-8624255983c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:08.565681 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5e0a3-a0d7-4231-95d1-8624255983c0 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:08.576822 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1295/2595] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:02:09.484587 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f4e27-2ee2-428c-95b8-2198082e84a6 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:09.486104 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-731f4e27-2ee2-428c-95b8-2198082e84a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:09.505851 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f4e27-2ee2-428c-95b8-2198082e84a6 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:09.523646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8876a80-c882-4e78-ba50-f63c0bcc7812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:09.524393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8876a80-c882-4e78-ba50-f63c0bcc7812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:09.527696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8876a80-c882-4e78-ba50-f63c0bcc7812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:09.527696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8876a80-c882-4e78-ba50-f63c0bcc7812 admin admin] PATH_INFO: `/v3/projects/9e7cdb81067d4e1dbdb3feecbc785504` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:09.527696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8876a80-c882-4e78-ba50-f63c0bcc7812 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e7cdb81067d4e1dbdb3feecbc785504)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:09.528239 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8876a80-c882-4e78-ba50-f63c0bcc7812 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:09.562537 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:09.647405 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b8876a80-c882-4e78-ba50-f63c0bcc7812 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e7cdb81067d4e1dbdb3feecbc785504', 'request_id': 'req-b8876a80-c882-4e78-ba50-f63c0bcc7812'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:09.737039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b8876a80-c882-4e78-ba50-f63c0bcc7812 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e7cdb81067d4e1dbdb3feecbc785504', 'request_id': 'req-b8876a80-c882-4e78-ba50-f63c0bcc7812'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:09.747075 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b8876a80-c882-4e78-ba50-f63c0bcc7812 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e7cdb81067d4e1dbdb3feecbc785504', 'request_id': 'req-b8876a80-c882-4e78-ba50-f63c0bcc7812'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:09.755438 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b8876a80-c882-4e78-ba50-f63c0bcc7812 admin admin] The token cache is being invalidate because project 9e7cdb81067d4e1dbdb3feecbc785504 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:02:09.755729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b8876a80-c882-4e78-ba50-f63c0bcc7812 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:09.756384 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b8876a80-c882-4e78-ba50-f63c0bcc7812 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:09.758103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1301/2596] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:02:09 2022] DELETE /identity/v3/projects/9e7cdb81067d4e1dbdb3feecbc785504 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:09.762292 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5e0a3-a0d7-4231-95d1-8624255983c0 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:09.772831 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5e0a3-a0d7-4231-95d1-8624255983c0 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:09.828498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1772a0a-4a00-4187-ada4-ccad238469b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:09.829162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1772a0a-4a00-4187-ada4-ccad238469b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:09.829442 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1772a0a-4a00-4187-ada4-ccad238469b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:09.829713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1772a0a-4a00-4187-ada4-ccad238469b5 admin admin] PATH_INFO: `/v3/users/be44a831be4b4f25a778495b19b1e800` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:09.830843 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1772a0a-4a00-4187-ada4-ccad238469b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be44a831be4b4f25a778495b19b1e800)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:09.832794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1772a0a-4a00-4187-ada4-ccad238469b5 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:10.034497 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e1772a0a-4a00-4187-ada4-ccad238469b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be44a831be4b4f25a778495b19b1e800'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:10.048231 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e1772a0a-4a00-4187-ada4-ccad238469b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be44a831be4b4f25a778495b19b1e800'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:10.055571 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e1772a0a-4a00-4187-ada4-ccad238469b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be44a831be4b4f25a778495b19b1e800'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:10.159693 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e1772a0a-4a00-4187-ada4-ccad238469b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be44a831be4b4f25a778495b19b1e800'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:10.160346 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e1772a0a-4a00-4187-ada4-ccad238469b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be44a831be4b4f25a778495b19b1e800'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:10.167485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1296/2597] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:02:09 2022] DELETE /identity/v3/users/be44a831be4b4f25a778495b19b1e800 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:10.621064 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8876a80-c882-4e78-ba50-f63c0bcc7812 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:10.622545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b8876a80-c882-4e78-ba50-f63c0bcc7812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:10.636406 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8876a80-c882-4e78-ba50-f63c0bcc7812 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:10.697069 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6650de8-5955-4f8c-93fd-db52e172fac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:10.697627 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6650de8-5955-4f8c-93fd-db52e172fac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:10.697911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6650de8-5955-4f8c-93fd-db52e172fac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:10.698191 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6650de8-5955-4f8c-93fd-db52e172fac0 admin admin] PATH_INFO: `/v3/projects/38bda36e8f2c438fa1cb56228e16db1f` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:10.699363 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6650de8-5955-4f8c-93fd-db52e172fac0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38bda36e8f2c438fa1cb56228e16db1f)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:10.701271 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6650de8-5955-4f8c-93fd-db52e172fac0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:10.806386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f6650de8-5955-4f8c-93fd-db52e172fac0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38bda36e8f2c438fa1cb56228e16db1f', 'request_id': 'req-f6650de8-5955-4f8c-93fd-db52e172fac0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:10.881982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f6650de8-5955-4f8c-93fd-db52e172fac0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38bda36e8f2c438fa1cb56228e16db1f', 'request_id': 'req-f6650de8-5955-4f8c-93fd-db52e172fac0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:10.890985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f6650de8-5955-4f8c-93fd-db52e172fac0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38bda36e8f2c438fa1cb56228e16db1f', 'request_id': 'req-f6650de8-5955-4f8c-93fd-db52e172fac0'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:10.903084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f6650de8-5955-4f8c-93fd-db52e172fac0 admin admin] The token cache is being invalidate because project 38bda36e8f2c438fa1cb56228e16db1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:02:10.903408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f6650de8-5955-4f8c-93fd-db52e172fac0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:10.904270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-f6650de8-5955-4f8c-93fd-db52e172fac0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:10.906013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1302/2598] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:02:10 2022] DELETE /identity/v3/projects/38bda36e8f2c438fa1cb56228e16db1f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:10.911321 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fe24b-ded3-4c16-848a-0914216a8504 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:10.911787 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fe24b-ded3-4c16-848a-0914216a8504 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:10.911903 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fe24b-ded3-4c16-848a-0914216a8504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:10.932048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-7f5fe24b-ded3-4c16-848a-0914216a8504 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:10.933256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7f5fe24b-ded3-4c16-848a-0914216a8504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:10.938415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:10.993776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1297/2599] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:11.001462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e9254-28d2-4193-ae85-972d3dba2c62 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:11.001746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e9254-28d2-4193-ae85-972d3dba2c62 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:11.002021 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e9254-28d2-4193-ae85-972d3dba2c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:11.023372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-882e9254-28d2-4193-ae85-972d3dba2c62 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:11.024595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-882e9254-28d2-4193-ae85-972d3dba2c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:11.032904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1303/2600] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:11.037157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fe24b-ded3-4c16-848a-0914216a8504 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:11.046743 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fe24b-ded3-4c16-848a-0914216a8504 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:11.051333 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab9a6a-e994-40df-a44d-cbcb9416a41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:11.051926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab9a6a-e994-40df-a44d-cbcb9416a41c admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:11.052496 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab9a6a-e994-40df-a44d-cbcb9416a41c admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:11.052774 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab9a6a-e994-40df-a44d-cbcb9416a41c admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:11.053429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab9a6a-e994-40df-a44d-cbcb9416a41c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:02:11.053732 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab9a6a-e994-40df-a44d-cbcb9416a41c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:11.055372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab9a6a-e994-40df-a44d-cbcb9416a41c admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:11.067666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1298/2601] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:02:11.072491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e9254-28d2-4193-ae85-972d3dba2c62 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:11.081676 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e9254-28d2-4193-ae85-972d3dba2c62 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:11.087129 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d184c5-c1c6-4e60-87f1-c5512d46ab89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:11.088314 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d184c5-c1c6-4e60-87f1-c5512d46ab89 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:11.088314 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d184c5-c1c6-4e60-87f1-c5512d46ab89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:11.088314 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d184c5-c1c6-4e60-87f1-c5512d46ab89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:11.088946 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d184c5-c1c6-4e60-87f1-c5512d46ab89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:11.090871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d184c5-c1c6-4e60-87f1-c5512d46ab89 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:11.186929 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1304/2602] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:02:11 2022] POST /identity/v3/projects => generated 367 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:02:11.191122 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab9a6a-e994-40df-a44d-cbcb9416a41c admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:11.201670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab9a6a-e994-40df-a44d-cbcb9416a41c admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:11.261656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e3ab1-aae8-4547-882d-c94b622ef93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:11.262276 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e3ab1-aae8-4547-882d-c94b622ef93b admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:11.262561 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e3ab1-aae8-4547-882d-c94b622ef93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:11.262831 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e3ab1-aae8-4547-882d-c94b622ef93b admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:11.263505 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e3ab1-aae8-4547-882d-c94b622ef93b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:11.268466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e3ab1-aae8-4547-882d-c94b622ef93b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:11.356256 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1299/2603] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:02:11 2022] POST /identity/v3/users => generated 284 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:02:11.360408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d184c5-c1c6-4e60-87f1-c5512d46ab89 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:11.371838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d184c5-c1c6-4e60-87f1-c5512d46ab89 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:11.376532 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3bc4c-aacf-4cbd-a67e-b5d513b38a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:11.377152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3bc4c-aacf-4cbd-a67e-b5d513b38a3d admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:11.377644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3bc4c-aacf-4cbd-a67e-b5d513b38a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:11.377932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3bc4c-aacf-4cbd-a67e-b5d513b38a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:11.378582 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3bc4c-aacf-4cbd-a67e-b5d513b38a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:11.380273 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3bc4c-aacf-4cbd-a67e-b5d513b38a3d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:11.382255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:11.391803 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1305/2604] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:02:11.396157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e3ab1-aae8-4547-882d-c94b622ef93b admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:11.406596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e3ab1-aae8-4547-882d-c94b622ef93b admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:11.411483 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec1a57-fe4c-48b5-bf84-5818c8107462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:11.412094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec1a57-fe4c-48b5-bf84-5818c8107462 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:11.412375 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec1a57-fe4c-48b5-bf84-5818c8107462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:11.412721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec1a57-fe4c-48b5-bf84-5818c8107462 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:11.413365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec1a57-fe4c-48b5-bf84-5818c8107462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:11.414971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec1a57-fe4c-48b5-bf84-5818c8107462 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:11.426197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1300/2605] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:02:11.431009 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3bc4c-aacf-4cbd-a67e-b5d513b38a3d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:11.441054 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3bc4c-aacf-4cbd-a67e-b5d513b38a3d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:11.446280 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6d6cd-3b0b-4960-bcf4-9a15dc1ee009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:11.446942 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6d6cd-3b0b-4960-bcf4-9a15dc1ee009 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:11.447345 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6d6cd-3b0b-4960-bcf4-9a15dc1ee009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:11.447621 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6d6cd-3b0b-4960-bcf4-9a15dc1ee009 admin admin] PATH_INFO: `/v3/projects/db50edd6059e4d7cb76c0542d2b88151/users/208f1ad67dab4aa0a4b54f7bc3ad39c9/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:11.463199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6d6cd-3b0b-4960-bcf4-9a15dc1ee009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db50edd6059e4d7cb76c0542d2b88151, user_id=208f1ad67dab4aa0a4b54f7bc3ad39c9, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:11.465933 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6d6cd-3b0b-4960-bcf4-9a15dc1ee009 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:11.588820 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1306/2606] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:02:11 2022] PUT /identity/v3/projects/db50edd6059e4d7cb76c0542d2b88151/users/208f1ad67dab4aa0a4b54f7bc3ad39c9/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:02:11.599350 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcfa80-4ad9-4e5f-88dc-dc8be3e8038f None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:11.599632 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcfa80-4ad9-4e5f-88dc-dc8be3e8038f None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:11.599913 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcfa80-4ad9-4e5f-88dc-dc8be3e8038f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:11.628602 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-68fcfa80-4ad9-4e5f-88dc-dc8be3e8038f None None] MFA Rules not processed for user `208f1ad67dab4aa0a4b54f7bc3ad39c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:11.629794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-68fcfa80-4ad9-4e5f-88dc-dc8be3e8038f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:11.693670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1301/2607] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:11.698298 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6d6cd-3b0b-4960-bcf4-9a15dc1ee009 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:11.763882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6d6cd-3b0b-4960-bcf4-9a15dc1ee009 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:11.768800 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34402b95-e272-4128-b757-3a2fc2897ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:11.769453 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34402b95-e272-4128-b757-3a2fc2897ef5 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:11.770001 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34402b95-e272-4128-b757-3a2fc2897ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:11.770282 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34402b95-e272-4128-b757-3a2fc2897ef5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:11.770939 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34402b95-e272-4128-b757-3a2fc2897ef5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:11.773014 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34402b95-e272-4128-b757-3a2fc2897ef5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:11.899076 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1307/2608] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:02:11 2022] POST /identity/v3/projects => generated 369 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:02:11.900335 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fcfa80-4ad9-4e5f-88dc-dc8be3e8038f None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:11.911739 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fcfa80-4ad9-4e5f-88dc-dc8be3e8038f None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:11.916060 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:11.971894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cf380-678d-4919-9e1b-fbf058eb082a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:11.972509 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cf380-678d-4919-9e1b-fbf058eb082a admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:11.972786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cf380-678d-4919-9e1b-fbf058eb082a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:11.973058 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cf380-678d-4919-9e1b-fbf058eb082a admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:11.973730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363cf380-678d-4919-9e1b-fbf058eb082a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:11.976290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363cf380-678d-4919-9e1b-fbf058eb082a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:12.065192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1302/2609] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:02:11 2022] POST /identity/v3/users => generated 291 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:02:12.069163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34402b95-e272-4128-b757-3a2fc2897ef5 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:12.079875 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34402b95-e272-4128-b757-3a2fc2897ef5 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:12.085015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98617edf-a5b7-402e-a1ac-d4f46807bdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:12.085616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98617edf-a5b7-402e-a1ac-d4f46807bdf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:12.085901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98617edf-a5b7-402e-a1ac-d4f46807bdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:12.086208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98617edf-a5b7-402e-a1ac-d4f46807bdf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:12.086897 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98617edf-a5b7-402e-a1ac-d4f46807bdf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:12.088595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98617edf-a5b7-402e-a1ac-d4f46807bdf0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:12.099374 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1308/2610] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:02:12.104291 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cf380-678d-4919-9e1b-fbf058eb082a admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:12.114787 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cf380-678d-4919-9e1b-fbf058eb082a admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:12.120123 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b73685-3b4c-4c39-a2a4-735a2247694e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:12.120851 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b73685-3b4c-4c39-a2a4-735a2247694e admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:12.121133 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b73685-3b4c-4c39-a2a4-735a2247694e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:12.121406 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b73685-3b4c-4c39-a2a4-735a2247694e admin admin] PATH_INFO: `/v3/projects/63c4e146e4fd4e6a8aa72f0e8e0ea87a/users/e5bc277df780473f8c088343d508e2bd/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:12.136883 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b73685-3b4c-4c39-a2a4-735a2247694e admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c4e146e4fd4e6a8aa72f0e8e0ea87a, user_id=e5bc277df780473f8c088343d508e2bd, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:12.142533 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b73685-3b4c-4c39-a2a4-735a2247694e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:12.231826 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1303/2611] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:02:12 2022] PUT /identity/v3/projects/63c4e146e4fd4e6a8aa72f0e8e0ea87a/users/e5bc277df780473f8c088343d508e2bd/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:02:12.235845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98617edf-a5b7-402e-a1ac-d4f46807bdf0 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:12.245911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98617edf-a5b7-402e-a1ac-d4f46807bdf0 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:12.304084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f1e70-4d35-4f78-ab93-ca001066b52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:12.304697 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f1e70-4d35-4f78-ab93-ca001066b52f admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:12.304982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f1e70-4d35-4f78-ab93-ca001066b52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:12.305258 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f1e70-4d35-4f78-ab93-ca001066b52f admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:12.305905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f1e70-4d35-4f78-ab93-ca001066b52f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:12.307569 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f1e70-4d35-4f78-ab93-ca001066b52f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:12.317437 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1309/2612] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:02:12.321799 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b73685-3b4c-4c39-a2a4-735a2247694e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:12.332206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b73685-3b4c-4c39-a2a4-735a2247694e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:12.337778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c24d23-e078-45a3-97c1-991ac33f415c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:12.338814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c24d23-e078-45a3-97c1-991ac33f415c admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:12.339097 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c24d23-e078-45a3-97c1-991ac33f415c admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:12.339754 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c24d23-e078-45a3-97c1-991ac33f415c admin admin] PATH_INFO: `/v3/projects/63c4e146e4fd4e6a8aa72f0e8e0ea87a/users/e5bc277df780473f8c088343d508e2bd/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:12.342059 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c24d23-e078-45a3-97c1-991ac33f415c admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c4e146e4fd4e6a8aa72f0e8e0ea87a, user_id=e5bc277df780473f8c088343d508e2bd, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:12.345220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c24d23-e078-45a3-97c1-991ac33f415c admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:12.356296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1304/2613] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:02:12 2022] PUT /identity/v3/projects/63c4e146e4fd4e6a8aa72f0e8e0ea87a/users/e5bc277df780473f8c088343d508e2bd/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:12.367762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7202eb4-bc68-4a9e-95f7-612f839caab8 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:12.368062 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7202eb4-bc68-4a9e-95f7-612f839caab8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:12.368339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7202eb4-bc68-4a9e-95f7-612f839caab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:12.398510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d7202eb4-bc68-4a9e-95f7-612f839caab8 None None] MFA Rules not processed for user `e5bc277df780473f8c088343d508e2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:12.399769 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d7202eb4-bc68-4a9e-95f7-612f839caab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:12.433342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:12.470383 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1310/2614] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:12.481585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c24d23-e078-45a3-97c1-991ac33f415c admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:12.484876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c5c24d23-e078-45a3-97c1-991ac33f415c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:12.539881 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c24d23-e078-45a3-97c1-991ac33f415c admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:12.543465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914ed17-6e18-42de-8dfa-442ae1186b25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:12.544061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914ed17-6e18-42de-8dfa-442ae1186b25 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:12.544332 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914ed17-6e18-42de-8dfa-442ae1186b25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:12.544600 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914ed17-6e18-42de-8dfa-442ae1186b25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:12.553444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914ed17-6e18-42de-8dfa-442ae1186b25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:12.555526 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914ed17-6e18-42de-8dfa-442ae1186b25 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:12.564721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1305/2615] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:02:16.125869 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7202eb4-bc68-4a9e-95f7-612f839caab8 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:16.190079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7202eb4-bc68-4a9e-95f7-612f839caab8 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:16.194981 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347647f1-0ed8-4530-b756-5841639e54f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:16.195651 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347647f1-0ed8-4530-b756-5841639e54f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:16.195978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347647f1-0ed8-4530-b756-5841639e54f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:16.196220 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347647f1-0ed8-4530-b756-5841639e54f1 admin admin] PATH_INFO: `/v3/users/208f1ad67dab4aa0a4b54f7bc3ad39c9` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:16.197358 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347647f1-0ed8-4530-b756-5841639e54f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=208f1ad67dab4aa0a4b54f7bc3ad39c9)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:16.199328 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347647f1-0ed8-4530-b756-5841639e54f1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:16.410461 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-347647f1-0ed8-4530-b756-5841639e54f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '208f1ad67dab4aa0a4b54f7bc3ad39c9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:16.423736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-347647f1-0ed8-4530-b756-5841639e54f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '208f1ad67dab4aa0a4b54f7bc3ad39c9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:16.430738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-347647f1-0ed8-4530-b756-5841639e54f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '208f1ad67dab4aa0a4b54f7bc3ad39c9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:16.493047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-347647f1-0ed8-4530-b756-5841639e54f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '208f1ad67dab4aa0a4b54f7bc3ad39c9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:16.493673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-347647f1-0ed8-4530-b756-5841639e54f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '208f1ad67dab4aa0a4b54f7bc3ad39c9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:16.499956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1311/2616] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:02:16 2022] DELETE /identity/v3/users/208f1ad67dab4aa0a4b54f7bc3ad39c9 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:16.504536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401643e-44bb-4ed1-81e6-d7e010b9bed3 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:16.505017 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401643e-44bb-4ed1-81e6-d7e010b9bed3 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:16.505334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401643e-44bb-4ed1-81e6-d7e010b9bed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:16.525426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-e401643e-44bb-4ed1-81e6-d7e010b9bed3 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:16.526641 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e401643e-44bb-4ed1-81e6-d7e010b9bed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:16.556953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347647f1-0ed8-4530-b756-5841639e54f1 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:16.558894 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-347647f1-0ed8-4530-b756-5841639e54f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:16.586812 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1306/2617] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:16.596871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401643e-44bb-4ed1-81e6-d7e010b9bed3 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:16.598424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e401643e-44bb-4ed1-81e6-d7e010b9bed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:16.603452 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:16.611015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:16.637586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347647f1-0ed8-4530-b756-5841639e54f1 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:16.646782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff35193-2bae-483b-8ca6-6e0615fba2d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:16.646782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff35193-2bae-483b-8ca6-6e0615fba2d9 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:16.646782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff35193-2bae-483b-8ca6-6e0615fba2d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:16.646782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff35193-2bae-483b-8ca6-6e0615fba2d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:16.648931 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0ff35193-2bae-483b-8ca6-6e0615fba2d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:16.669372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401643e-44bb-4ed1-81e6-d7e010b9bed3 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:16.669372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d267d-a0d5-4274-91b5-bf0a7947c7f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:16.669372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d267d-a0d5-4274-91b5-bf0a7947c7f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:16.669372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d267d-a0d5-4274-91b5-bf0a7947c7f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:16.669806 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d267d-a0d5-4274-91b5-bf0a7947c7f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:16.680893 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d267d-a0d5-4274-91b5-bf0a7947c7f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:16.682499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d267d-a0d5-4274-91b5-bf0a7947c7f6 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:16.693241 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1307/2618] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:02:16.723312 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff35193-2bae-483b-8ca6-6e0615fba2d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:16.724925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff35193-2bae-483b-8ca6-6e0615fba2d9 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:16.734726 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1312/2619] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:02:17.095764 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d267d-a0d5-4274-91b5-bf0a7947c7f6 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:17.098593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ea2d267d-a0d5-4274-91b5-bf0a7947c7f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:17.133667 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d267d-a0d5-4274-91b5-bf0a7947c7f6 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:17.138868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:17.139573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:17.139852 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:17.140217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7 admin admin] PATH_INFO: `/v3/projects/db50edd6059e4d7cb76c0542d2b88151` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:17.141638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db50edd6059e4d7cb76c0542d2b88151)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:17.143718 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:17.260060 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db50edd6059e4d7cb76c0542d2b88151', 'request_id': 'req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:17.265647 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db50edd6059e4d7cb76c0542d2b88151', 'request_id': 'req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:17.350237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db50edd6059e4d7cb76c0542d2b88151', 'request_id': 'req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:17.363131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7 admin admin] The token cache is being invalidate because project db50edd6059e4d7cb76c0542d2b88151 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:02:17.364155 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:17.364155 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:17.365631 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1308/2620] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:02:17 2022] DELETE /identity/v3/projects/db50edd6059e4d7cb76c0542d2b88151 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:17.369790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff35193-2bae-483b-8ca6-6e0615fba2d9 service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:17.380773 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff35193-2bae-483b-8ca6-6e0615fba2d9 service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:17.444094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d14437-667d-47e9-a9c4-1d015bda4399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:17.444810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d14437-667d-47e9-a9c4-1d015bda4399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:17.445207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d14437-667d-47e9-a9c4-1d015bda4399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:17.445489 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d14437-667d-47e9-a9c4-1d015bda4399 admin admin] PATH_INFO: `/v3/users/e5bc277df780473f8c088343d508e2bd` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:17.446637 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d14437-667d-47e9-a9c4-1d015bda4399 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5bc277df780473f8c088343d508e2bd)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:17.448622 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d14437-667d-47e9-a9c4-1d015bda4399 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:17.608424 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a7d14437-667d-47e9-a9c4-1d015bda4399 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5bc277df780473f8c088343d508e2bd'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:17.620315 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a7d14437-667d-47e9-a9c4-1d015bda4399 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5bc277df780473f8c088343d508e2bd'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:17.627656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a7d14437-667d-47e9-a9c4-1d015bda4399 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5bc277df780473f8c088343d508e2bd'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:17.629356 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:17.697262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a7d14437-667d-47e9-a9c4-1d015bda4399 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5bc277df780473f8c088343d508e2bd'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:17.697817 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a7d14437-667d-47e9-a9c4-1d015bda4399 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5bc277df780473f8c088343d508e2bd'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:17.704119 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1313/2621] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:02:17 2022] DELETE /identity/v3/users/e5bc277df780473f8c088343d508e2bd => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:02:18.208750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:18.210279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:18.223940 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef02c9-c17e-4c9a-a24c-7364746bc0a7 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:18.284612 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e928b9e-5d6b-4c54-9af8-cfc688aecbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:18.285167 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e928b9e-5d6b-4c54-9af8-cfc688aecbf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:18.285446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e928b9e-5d6b-4c54-9af8-cfc688aecbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:18.285717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e928b9e-5d6b-4c54-9af8-cfc688aecbf3 admin admin] PATH_INFO: `/v3/projects/63c4e146e4fd4e6a8aa72f0e8e0ea87a` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:18.286883 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e928b9e-5d6b-4c54-9af8-cfc688aecbf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63c4e146e4fd4e6a8aa72f0e8e0ea87a)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:18.288818 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e928b9e-5d6b-4c54-9af8-cfc688aecbf3 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:18.381675 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:18.395819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1e928b9e-5d6b-4c54-9af8-cfc688aecbf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63c4e146e4fd4e6a8aa72f0e8e0ea87a', 'request_id': 'req-1e928b9e-5d6b-4c54-9af8-cfc688aecbf3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:18.401197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1e928b9e-5d6b-4c54-9af8-cfc688aecbf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63c4e146e4fd4e6a8aa72f0e8e0ea87a', 'request_id': 'req-1e928b9e-5d6b-4c54-9af8-cfc688aecbf3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:18.462306 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1e928b9e-5d6b-4c54-9af8-cfc688aecbf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63c4e146e4fd4e6a8aa72f0e8e0ea87a', 'request_id': 'req-1e928b9e-5d6b-4c54-9af8-cfc688aecbf3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:18.474492 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1e928b9e-5d6b-4c54-9af8-cfc688aecbf3 admin admin] The token cache is being invalidate because project 63c4e146e4fd4e6a8aa72f0e8e0ea87a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:02:18.474810 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1e928b9e-5d6b-4c54-9af8-cfc688aecbf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:18.475466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1e928b9e-5d6b-4c54-9af8-cfc688aecbf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:18.477455 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1309/2622] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:02:18 2022] DELETE /identity/v3/projects/63c4e146e4fd4e6a8aa72f0e8e0ea87a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:18.482438 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f1e64-73c0-4a46-9dc2-d96b5ace0c8e None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:18.482892 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f1e64-73c0-4a46-9dc2-d96b5ace0c8e None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:18.482975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f1e64-73c0-4a46-9dc2-d96b5ace0c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:18.503403 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f85f1e64-73c0-4a46-9dc2-d96b5ace0c8e None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:18.504606 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f85f1e64-73c0-4a46-9dc2-d96b5ace0c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:18.584877 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1314/2623] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:18.592728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b1594-abb8-4d0c-a003-399251875186 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:18.593014 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b1594-abb8-4d0c-a003-399251875186 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:18.593295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b1594-abb8-4d0c-a003-399251875186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:18.613357 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-920b1594-abb8-4d0c-a003-399251875186 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:18.614731 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-920b1594-abb8-4d0c-a003-399251875186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:18.623516 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1310/2624] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:18.627829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f1e64-73c0-4a46-9dc2-d96b5ace0c8e None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:18.637745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f1e64-73c0-4a46-9dc2-d96b5ace0c8e None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:18.642597 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f400c9b-8feb-4814-9156-87819067bcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:18.643262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f400c9b-8feb-4814-9156-87819067bcaa admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:18.643495 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f400c9b-8feb-4814-9156-87819067bcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:18.643767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f400c9b-8feb-4814-9156-87819067bcaa admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:18.644725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f400c9b-8feb-4814-9156-87819067bcaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:02:18.644984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f400c9b-8feb-4814-9156-87819067bcaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:18.646592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f400c9b-8feb-4814-9156-87819067bcaa admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:18.650964 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ee0f2-8d77-4e10-93d3-1c90eda07f41 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:18.651281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ee0f2-8d77-4e10-93d3-1c90eda07f41 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:18.651560 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ee0f2-8d77-4e10-93d3-1c90eda07f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:18.659903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1315/2625] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:02:18.664817 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f400c9b-8feb-4814-9156-87819067bcaa admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:18.674737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-bf9ee0f2-8d77-4e10-93d3-1c90eda07f41 None None] MFA Rules not processed for user `5a667fa07b1148998959835d9995ad60`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:18.675497 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f400c9b-8feb-4814-9156-87819067bcaa admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:18.676021 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-bf9ee0f2-8d77-4e10-93d3-1c90eda07f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:18.681033 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d4347-1b1b-410f-be79-1c9ee08340e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:18.681557 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d4347-1b1b-410f-be79-1c9ee08340e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:18.681834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d4347-1b1b-410f-be79-1c9ee08340e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:18.682103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d4347-1b1b-410f-be79-1c9ee08340e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:18.682734 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d4347-1b1b-410f-be79-1c9ee08340e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:18.684665 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d4347-1b1b-410f-be79-1c9ee08340e4 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:18.735619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1311/2626] 10.222.0.78 () {58 vars in 967 bytes} [Mon Sep 19 12:02:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:18.749440 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ee0f2-8d77-4e10-93d3-1c90eda07f41 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:18.750806 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-bf9ee0f2-8d77-4e10-93d3-1c90eda07f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:18.812577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1316/2627] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:02:18 2022] POST /identity/v3/projects => generated 381 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:02:18.816702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d4347-1b1b-410f-be79-1c9ee08340e4 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:18.820530 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ee0f2-8d77-4e10-93d3-1c90eda07f41 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:18.878283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:18.881282 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d4347-1b1b-410f-be79-1c9ee08340e4 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:18.885396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61d57d-dc79-4a77-b2c5-443c5411f7c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf462b2aa3fc4e2ca58851549cf0c644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:18.886002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05633d3a-766a-4ac8-89ef-ffac00604685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:18.886118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61d57d-dc79-4a77-b2c5-443c5411f7c4 service swift] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:18.886276 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61d57d-dc79-4a77-b2c5-443c5411f7c4 service swift] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:18.886543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61d57d-dc79-4a77-b2c5-443c5411f7c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:18.886641 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05633d3a-766a-4ac8-89ef-ffac00604685 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:18.886881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05633d3a-766a-4ac8-89ef-ffac00604685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:18.887157 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05633d3a-766a-4ac8-89ef-ffac00604685 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:18.887816 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05633d3a-766a-4ac8-89ef-ffac00604685 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:18.889747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05633d3a-766a-4ac8-89ef-ffac00604685 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:18.944144 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61d57d-dc79-4a77-b2c5-443c5411f7c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:18.945742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61d57d-dc79-4a77-b2c5-443c5411f7c4 service swift] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:18.955880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1312/2628] 10.222.0.78 () {60 vars in 1384 bytes} [Mon Sep 19 12:02:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 12:02:19.013623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1317/2629] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:02:18 2022] POST /identity/v3/users => generated 291 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:02:19.017762 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61d57d-dc79-4a77-b2c5-443c5411f7c4 service swift] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:19.029872 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61d57d-dc79-4a77-b2c5-443c5411f7c4 service swift] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:19.034562 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fc62a-6bb4-458d-abc2-120033a1db9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:19.035149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fc62a-6bb4-458d-abc2-120033a1db9e admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:19.035454 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fc62a-6bb4-458d-abc2-120033a1db9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:19.035725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fc62a-6bb4-458d-abc2-120033a1db9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:19.036361 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5fc62a-6bb4-458d-abc2-120033a1db9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:19.037945 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5fc62a-6bb4-458d-abc2-120033a1db9e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:19.048919 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1313/2630] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:02:19.053103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05633d3a-766a-4ac8-89ef-ffac00604685 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:19.063387 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05633d3a-766a-4ac8-89ef-ffac00604685 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:19.069486 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7df618-b925-4e76-a821-65a98bde7a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:19.070329 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7df618-b925-4e76-a821-65a98bde7a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:19.070686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7df618-b925-4e76-a821-65a98bde7a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:19.071040 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7df618-b925-4e76-a821-65a98bde7a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:19.071797 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7df618-b925-4e76-a821-65a98bde7a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:19.073577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7df618-b925-4e76-a821-65a98bde7a00 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:19.084566 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1318/2631] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:02:19.089445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fc62a-6bb4-458d-abc2-120033a1db9e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:19.099485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fc62a-6bb4-458d-abc2-120033a1db9e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:19.106364 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e22bc5-b35e-4485-85ab-fbf57269d438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:19.107038 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e22bc5-b35e-4485-85ab-fbf57269d438 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:19.107444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e22bc5-b35e-4485-85ab-fbf57269d438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:19.107607 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e22bc5-b35e-4485-85ab-fbf57269d438 admin admin] PATH_INFO: `/v3/projects/af8a434fa7cc461c93ab909fc6747b8d/users/0ed47d0496bf4ee2ac8d708ab772f6ca/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:19.123419 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e22bc5-b35e-4485-85ab-fbf57269d438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af8a434fa7cc461c93ab909fc6747b8d, user_id=0ed47d0496bf4ee2ac8d708ab772f6ca, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:19.126084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e22bc5-b35e-4485-85ab-fbf57269d438 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:19.211312 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1314/2632] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:02:19 2022] PUT /identity/v3/projects/af8a434fa7cc461c93ab909fc6747b8d/users/0ed47d0496bf4ee2ac8d708ab772f6ca/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:19.219543 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f9405-1e59-44ee-94b9-1cd80de078db None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:19.219858 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f9405-1e59-44ee-94b9-1cd80de078db None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:19.220327 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f9405-1e59-44ee-94b9-1cd80de078db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:19.251648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-1a5f9405-1e59-44ee-94b9-1cd80de078db None None] MFA Rules not processed for user `0ed47d0496bf4ee2ac8d708ab772f6ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:19.252874 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1a5f9405-1e59-44ee-94b9-1cd80de078db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:19.278801 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:19.319185 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1319/2633] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:19.324384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e22bc5-b35e-4485-85ab-fbf57269d438 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:19.395048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e22bc5-b35e-4485-85ab-fbf57269d438 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:19.401118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa9ceb-af4a-4c80-a824-a6c94201d2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:19.401723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa9ceb-af4a-4c80-a824-a6c94201d2d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:19.402008 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa9ceb-af4a-4c80-a824-a6c94201d2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:19.402284 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa9ceb-af4a-4c80-a824-a6c94201d2d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:19.402930 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa9ceb-af4a-4c80-a824-a6c94201d2d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:19.404926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa9ceb-af4a-4c80-a824-a6c94201d2d1 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:19.488760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1315/2634] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:02:19 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:02:19.493468 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f9405-1e59-44ee-94b9-1cd80de078db None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:19.504070 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f9405-1e59-44ee-94b9-1cd80de078db None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:19.562823 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b948a8b-0236-4488-af37-a41d8b018c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:19.563465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b948a8b-0236-4488-af37-a41d8b018c9f admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:19.563751 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b948a8b-0236-4488-af37-a41d8b018c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:19.564026 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b948a8b-0236-4488-af37-a41d8b018c9f admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:19.564674 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b948a8b-0236-4488-af37-a41d8b018c9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:19.566740 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b948a8b-0236-4488-af37-a41d8b018c9f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:19.693546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1320/2635] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:02:19 2022] POST /identity/v3/users => generated 295 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:02:19.697649 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa9ceb-af4a-4c80-a824-a6c94201d2d1 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:19.707814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa9ceb-af4a-4c80-a824-a6c94201d2d1 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:19.714047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3136db4-da63-4426-8238-4e7c3304d9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:19.714652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3136db4-da63-4426-8238-4e7c3304d9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:19.714930 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3136db4-da63-4426-8238-4e7c3304d9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:19.715229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3136db4-da63-4426-8238-4e7c3304d9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:19.715875 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3136db4-da63-4426-8238-4e7c3304d9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:19.717468 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3136db4-da63-4426-8238-4e7c3304d9a6 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:19.729950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1316/2636] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:02:19.734137 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b948a8b-0236-4488-af37-a41d8b018c9f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:19.744504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b948a8b-0236-4488-af37-a41d8b018c9f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:19.749434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b30a8-11b0-4f36-b8e0-0665d61ec105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:19.750096 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b30a8-11b0-4f36-b8e0-0665d61ec105 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:19.750378 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b30a8-11b0-4f36-b8e0-0665d61ec105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:19.750651 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b30a8-11b0-4f36-b8e0-0665d61ec105 admin admin] PATH_INFO: `/v3/projects/b6a6ce89d3c0415d99a2eb3b6106d271/users/dc7f47e810c244aba85bc6c0583d8001/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:19.766513 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b30a8-11b0-4f36-b8e0-0665d61ec105 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6a6ce89d3c0415d99a2eb3b6106d271, user_id=dc7f47e810c244aba85bc6c0583d8001, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:19.769218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b30a8-11b0-4f36-b8e0-0665d61ec105 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:19.848034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1321/2637] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:02:19 2022] PUT /identity/v3/projects/b6a6ce89d3c0415d99a2eb3b6106d271/users/dc7f47e810c244aba85bc6c0583d8001/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:19.852775 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3136db4-da63-4426-8238-4e7c3304d9a6 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:19.863574 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3136db4-da63-4426-8238-4e7c3304d9a6 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:19.904139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:19.922171 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5a93d-4472-4341-818b-b318435c4371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:19.922782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5a93d-4472-4341-818b-b318435c4371 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:19.923067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5a93d-4472-4341-818b-b318435c4371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:19.923371 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5a93d-4472-4341-818b-b318435c4371 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:19.924022 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5a93d-4472-4341-818b-b318435c4371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:19.925764 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5a93d-4472-4341-818b-b318435c4371 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:19.935782 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1317/2638] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:02:19.940151 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b30a8-11b0-4f36-b8e0-0665d61ec105 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:19.951028 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b30a8-11b0-4f36-b8e0-0665d61ec105 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:19.955744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0eff3-d83f-4b3f-93bd-df0451e2108d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:19.956406 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0eff3-d83f-4b3f-93bd-df0451e2108d admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:19.956690 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0eff3-d83f-4b3f-93bd-df0451e2108d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:19.956964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0eff3-d83f-4b3f-93bd-df0451e2108d admin admin] PATH_INFO: `/v3/projects/b6a6ce89d3c0415d99a2eb3b6106d271/users/dc7f47e810c244aba85bc6c0583d8001/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:19.958888 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0eff3-d83f-4b3f-93bd-df0451e2108d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6a6ce89d3c0415d99a2eb3b6106d271, user_id=dc7f47e810c244aba85bc6c0583d8001, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:19.962224 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0eff3-d83f-4b3f-93bd-df0451e2108d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:19.973198 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1322/2639] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:02:19 2022] PUT /identity/v3/projects/b6a6ce89d3c0415d99a2eb3b6106d271/users/dc7f47e810c244aba85bc6c0583d8001/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:19.984749 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf116b-0ad8-4a0a-b3e8-1a007e6d67da None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:19.985037 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf116b-0ad8-4a0a-b3e8-1a007e6d67da None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:19.985479 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf116b-0ad8-4a0a-b3e8-1a007e6d67da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:20.015159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-1edf116b-0ad8-4a0a-b3e8-1a007e6d67da None None] MFA Rules not processed for user `dc7f47e810c244aba85bc6c0583d8001`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:20.016404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1edf116b-0ad8-4a0a-b3e8-1a007e6d67da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:20.085565 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1318/2640] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:20.096715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0eff3-d83f-4b3f-93bd-df0451e2108d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:20.098474 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e7d0eff3-d83f-4b3f-93bd-df0451e2108d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:20.158001 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0eff3-d83f-4b3f-93bd-df0451e2108d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:20.161872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f09ea2-5ea6-48e1-a6e3-d1e46b170d8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:20.162546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f09ea2-5ea6-48e1-a6e3-d1e46b170d8b service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:20.162822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f09ea2-5ea6-48e1-a6e3-d1e46b170d8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:20.163101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f09ea2-5ea6-48e1-a6e3-d1e46b170d8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:20.172847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f09ea2-5ea6-48e1-a6e3-d1e46b170d8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:20.174312 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f09ea2-5ea6-48e1-a6e3-d1e46b170d8b service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:20.184226 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1323/2641] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:02:20.730803 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf116b-0ad8-4a0a-b3e8-1a007e6d67da None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:20.793564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf116b-0ad8-4a0a-b3e8-1a007e6d67da None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:20.797948 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5caf6-4800-4917-ba92-1bc74cf520c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:20.798568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5caf6-4800-4917-ba92-1bc74cf520c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:20.798849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5caf6-4800-4917-ba92-1bc74cf520c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:20.799126 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5caf6-4800-4917-ba92-1bc74cf520c4 admin admin] PATH_INFO: `/v3/users/0ed47d0496bf4ee2ac8d708ab772f6ca` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:20.800266 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5caf6-4800-4917-ba92-1bc74cf520c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ed47d0496bf4ee2ac8d708ab772f6ca)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:20.802169 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5caf6-4800-4917-ba92-1bc74cf520c4 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:20.979591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-23f5caf6-4800-4917-ba92-1bc74cf520c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ed47d0496bf4ee2ac8d708ab772f6ca'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:20.991050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:20.992987 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-23f5caf6-4800-4917-ba92-1bc74cf520c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ed47d0496bf4ee2ac8d708ab772f6ca'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:21.001381 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-23f5caf6-4800-4917-ba92-1bc74cf520c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ed47d0496bf4ee2ac8d708ab772f6ca'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:21.104252 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-23f5caf6-4800-4917-ba92-1bc74cf520c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ed47d0496bf4ee2ac8d708ab772f6ca'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:21.104827 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-23f5caf6-4800-4917-ba92-1bc74cf520c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ed47d0496bf4ee2ac8d708ab772f6ca'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:21.111696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1319/2642] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:02:20 2022] DELETE /identity/v3/users/0ed47d0496bf4ee2ac8d708ab772f6ca => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:21.116279 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92146540-3530-4199-937c-95396a6baea6 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:21.117410 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92146540-3530-4199-937c-95396a6baea6 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:21.117410 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92146540-3530-4199-937c-95396a6baea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:21.137408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-92146540-3530-4199-937c-95396a6baea6 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:21.138644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-92146540-3530-4199-937c-95396a6baea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:21.152722 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:21.200458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1324/2643] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:21.210080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5caf6-4800-4917-ba92-1bc74cf520c4 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:21.212045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-23f5caf6-4800-4917-ba92-1bc74cf520c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:21.271484 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5caf6-4800-4917-ba92-1bc74cf520c4 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:21.275897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863ad20-5285-4578-baf4-798738b07b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:21.276617 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863ad20-5285-4578-baf4-798738b07b0f service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:21.276827 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863ad20-5285-4578-baf4-798738b07b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:21.277117 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863ad20-5285-4578-baf4-798738b07b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:21.286850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863ad20-5285-4578-baf4-798738b07b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:21.288310 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863ad20-5285-4578-baf4-798738b07b0f service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:21.298709 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1320/2644] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:02:21.704523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92146540-3530-4199-937c-95396a6baea6 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:21.705956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-92146540-3530-4199-937c-95396a6baea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:21.719666 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92146540-3530-4199-937c-95396a6baea6 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:21.724359 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d743340-eb94-4593-83fb-b2b64e82ddd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:21.725000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d743340-eb94-4593-83fb-b2b64e82ddd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:21.725286 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d743340-eb94-4593-83fb-b2b64e82ddd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:21.725569 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d743340-eb94-4593-83fb-b2b64e82ddd5 admin admin] PATH_INFO: `/v3/projects/af8a434fa7cc461c93ab909fc6747b8d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:21.726747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d743340-eb94-4593-83fb-b2b64e82ddd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af8a434fa7cc461c93ab909fc6747b8d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:21.728807 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d743340-eb94-4593-83fb-b2b64e82ddd5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:21.847610 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6d743340-eb94-4593-83fb-b2b64e82ddd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af8a434fa7cc461c93ab909fc6747b8d', 'request_id': 'req-6d743340-eb94-4593-83fb-b2b64e82ddd5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:21.944476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6d743340-eb94-4593-83fb-b2b64e82ddd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af8a434fa7cc461c93ab909fc6747b8d', 'request_id': 'req-6d743340-eb94-4593-83fb-b2b64e82ddd5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:21.952535 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6d743340-eb94-4593-83fb-b2b64e82ddd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af8a434fa7cc461c93ab909fc6747b8d', 'request_id': 'req-6d743340-eb94-4593-83fb-b2b64e82ddd5'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:21.963526 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6d743340-eb94-4593-83fb-b2b64e82ddd5 admin admin] The token cache is being invalidate because project af8a434fa7cc461c93ab909fc6747b8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:02:21.963823 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6d743340-eb94-4593-83fb-b2b64e82ddd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:21.964478 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-6d743340-eb94-4593-83fb-b2b64e82ddd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:21.965980 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1325/2645] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:02:21 2022] DELETE /identity/v3/projects/af8a434fa7cc461c93ab909fc6747b8d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:21.970153 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863ad20-5285-4578-baf4-798738b07b0f service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:21.980009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863ad20-5285-4578-baf4-798738b07b0f service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:22.036796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9b43a-a15d-4f52-93be-d32f141cb640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:22.037433 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9b43a-a15d-4f52-93be-d32f141cb640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:22.037716 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9b43a-a15d-4f52-93be-d32f141cb640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:22.037989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9b43a-a15d-4f52-93be-d32f141cb640 admin admin] PATH_INFO: `/v3/users/dc7f47e810c244aba85bc6c0583d8001` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:22.039113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9b43a-a15d-4f52-93be-d32f141cb640 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc7f47e810c244aba85bc6c0583d8001)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:22.041069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9b43a-a15d-4f52-93be-d32f141cb640 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:22.330455 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a5e9b43a-a15d-4f52-93be-d32f141cb640 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc7f47e810c244aba85bc6c0583d8001'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:22.343196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a5e9b43a-a15d-4f52-93be-d32f141cb640 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc7f47e810c244aba85bc6c0583d8001'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:22.350529 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a5e9b43a-a15d-4f52-93be-d32f141cb640 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc7f47e810c244aba85bc6c0583d8001'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:22.430181 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a5e9b43a-a15d-4f52-93be-d32f141cb640 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc7f47e810c244aba85bc6c0583d8001'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:22.430836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a5e9b43a-a15d-4f52-93be-d32f141cb640 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc7f47e810c244aba85bc6c0583d8001'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:22.437480 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1321/2646] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:02:21 2022] DELETE /identity/v3/users/dc7f47e810c244aba85bc6c0583d8001 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:02:22.854465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d743340-eb94-4593-83fb-b2b64e82ddd5 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:22.856055 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6d743340-eb94-4593-83fb-b2b64e82ddd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:22.871468 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d743340-eb94-4593-83fb-b2b64e82ddd5 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:22.930256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89866b-e176-4927-bb5b-6aa65454ddfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:22.930971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89866b-e176-4927-bb5b-6aa65454ddfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:22.931293 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89866b-e176-4927-bb5b-6aa65454ddfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:22.931567 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89866b-e176-4927-bb5b-6aa65454ddfc admin admin] PATH_INFO: `/v3/projects/b6a6ce89d3c0415d99a2eb3b6106d271` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:22.932723 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89866b-e176-4927-bb5b-6aa65454ddfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6a6ce89d3c0415d99a2eb3b6106d271)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:22.934633 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89866b-e176-4927-bb5b-6aa65454ddfc admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:23.071436 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:23.074417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5f89866b-e176-4927-bb5b-6aa65454ddfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6a6ce89d3c0415d99a2eb3b6106d271', 'request_id': 'req-5f89866b-e176-4927-bb5b-6aa65454ddfc'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:23.184418 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5f89866b-e176-4927-bb5b-6aa65454ddfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6a6ce89d3c0415d99a2eb3b6106d271', 'request_id': 'req-5f89866b-e176-4927-bb5b-6aa65454ddfc'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:23.191558 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5f89866b-e176-4927-bb5b-6aa65454ddfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6a6ce89d3c0415d99a2eb3b6106d271', 'request_id': 'req-5f89866b-e176-4927-bb5b-6aa65454ddfc'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:23.201879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5f89866b-e176-4927-bb5b-6aa65454ddfc admin admin] The token cache is being invalidate because project b6a6ce89d3c0415d99a2eb3b6106d271 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:02:23.202169 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5f89866b-e176-4927-bb5b-6aa65454ddfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:23.202814 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-5f89866b-e176-4927-bb5b-6aa65454ddfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:23.204332 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1326/2647] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:02:22 2022] DELETE /identity/v3/projects/b6a6ce89d3c0415d99a2eb3b6106d271 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:23.209234 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366115f5-69b9-44bd-8044-20c2d15e3bf1 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:23.209725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366115f5-69b9-44bd-8044-20c2d15e3bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:23.209821 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366115f5-69b9-44bd-8044-20c2d15e3bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:23.217626 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:23.230307 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-366115f5-69b9-44bd-8044-20c2d15e3bf1 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:23.231537 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-366115f5-69b9-44bd-8044-20c2d15e3bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:23.299330 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1322/2648] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:23.307357 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78357d-3d44-4a13-92de-2acaa9a690d4 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:23.307833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78357d-3d44-4a13-92de-2acaa9a690d4 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:23.308042 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78357d-3d44-4a13-92de-2acaa9a690d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:23.328771 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-2d78357d-3d44-4a13-92de-2acaa9a690d4 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:23.330000 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2d78357d-3d44-4a13-92de-2acaa9a690d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:23.338464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1327/2649] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:23.344139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366115f5-69b9-44bd-8044-20c2d15e3bf1 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:23.355772 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366115f5-69b9-44bd-8044-20c2d15e3bf1 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:23.360333 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d169097-8b51-48c0-bf16-fcee4f8ed713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:23.360931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d169097-8b51-48c0-bf16-fcee4f8ed713 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:23.361227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d169097-8b51-48c0-bf16-fcee4f8ed713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:23.361500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d169097-8b51-48c0-bf16-fcee4f8ed713 admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:23.362138 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d169097-8b51-48c0-bf16-fcee4f8ed713 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:02:23.362436 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d169097-8b51-48c0-bf16-fcee4f8ed713 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:23.364050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d169097-8b51-48c0-bf16-fcee4f8ed713 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:23.375869 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1323/2650] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:02:23.386762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78357d-3d44-4a13-92de-2acaa9a690d4 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:23.397106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78357d-3d44-4a13-92de-2acaa9a690d4 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:23.402330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934f131-ae90-4bd2-8ff9-f608371ba7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:23.402972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934f131-ae90-4bd2-8ff9-f608371ba7e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:23.403286 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934f131-ae90-4bd2-8ff9-f608371ba7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:23.403555 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934f131-ae90-4bd2-8ff9-f608371ba7e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:23.404289 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934f131-ae90-4bd2-8ff9-f608371ba7e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:23.406176 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934f131-ae90-4bd2-8ff9-f608371ba7e4 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:23.477449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1328/2651] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:02:23 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:02:23.481834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d169097-8b51-48c0-bf16-fcee4f8ed713 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:23.491865 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d169097-8b51-48c0-bf16-fcee4f8ed713 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:23.548412 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d2620-7060-4807-96bf-027a7e12ce9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:23.549013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d2620-7060-4807-96bf-027a7e12ce9e admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:23.549293 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d2620-7060-4807-96bf-027a7e12ce9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:23.549565 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d2620-7060-4807-96bf-027a7e12ce9e admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:23.550200 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d2620-7060-4807-96bf-027a7e12ce9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:23.552150 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d2620-7060-4807-96bf-027a7e12ce9e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:23.642747 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1324/2652] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:02:23 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:02:23.649601 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934f131-ae90-4bd2-8ff9-f608371ba7e4 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:23.660628 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934f131-ae90-4bd2-8ff9-f608371ba7e4 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:23.665552 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e8445-f496-45ac-b980-f5dadf09ccd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:23.666166 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e8445-f496-45ac-b980-f5dadf09ccd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:23.666448 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e8445-f496-45ac-b980-f5dadf09ccd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:23.666721 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e8445-f496-45ac-b980-f5dadf09ccd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:23.667395 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e8445-f496-45ac-b980-f5dadf09ccd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:23.669273 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e8445-f496-45ac-b980-f5dadf09ccd6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:23.680754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1329/2653] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:02:23.686273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d2620-7060-4807-96bf-027a7e12ce9e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:23.697658 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d2620-7060-4807-96bf-027a7e12ce9e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:23.702290 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21c44b-d80b-42ea-9d69-72f03f870552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:23.702886 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21c44b-d80b-42ea-9d69-72f03f870552 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:23.703189 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21c44b-d80b-42ea-9d69-72f03f870552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:23.703474 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21c44b-d80b-42ea-9d69-72f03f870552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:23.704109 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21c44b-d80b-42ea-9d69-72f03f870552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:23.705729 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21c44b-d80b-42ea-9d69-72f03f870552 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:23.718006 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1325/2654] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:02:23.722702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e8445-f496-45ac-b980-f5dadf09ccd6 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:23.733134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e8445-f496-45ac-b980-f5dadf09ccd6 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:23.738013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10810a-1094-405f-9c59-bade999b724e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:23.738581 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10810a-1094-405f-9c59-bade999b724e admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:23.738862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10810a-1094-405f-9c59-bade999b724e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:23.739140 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10810a-1094-405f-9c59-bade999b724e admin admin] PATH_INFO: `/v3/projects/a82995e77f0e435c9d16d0d23affebb6/users/efebb7fe59f448fc8a827ac83877520a/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:23.754538 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10810a-1094-405f-9c59-bade999b724e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a82995e77f0e435c9d16d0d23affebb6, user_id=efebb7fe59f448fc8a827ac83877520a, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:23.757289 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10810a-1094-405f-9c59-bade999b724e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:23.876629 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1330/2655] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:02:23 2022] PUT /identity/v3/projects/a82995e77f0e435c9d16d0d23affebb6/users/efebb7fe59f448fc8a827ac83877520a/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:23.888425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd6db9-c54f-41c2-892b-904933b8b248 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:23.888725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd6db9-c54f-41c2-892b-904933b8b248 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:23.889035 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd6db9-c54f-41c2-892b-904933b8b248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:23.918194 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-07bd6db9-c54f-41c2-892b-904933b8b248 None None] MFA Rules not processed for user `efebb7fe59f448fc8a827ac83877520a`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:23.919362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-07bd6db9-c54f-41c2-892b-904933b8b248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:23.986668 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:23.992659 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1326/2656] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:24.003509 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10810a-1094-405f-9c59-bade999b724e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:24.005234 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6c10810a-1094-405f-9c59-bade999b724e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:24.069883 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10810a-1094-405f-9c59-bade999b724e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:24.077366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1d070-2598-442b-aab8-7a482607a4b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:24.077995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1d070-2598-442b-aab8-7a482607a4b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:24.078283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1d070-2598-442b-aab8-7a482607a4b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:24.078562 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1d070-2598-442b-aab8-7a482607a4b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:24.087709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1d070-2598-442b-aab8-7a482607a4b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:24.089139 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1d070-2598-442b-aab8-7a482607a4b8 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:24.098303 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1331/2657] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:02:26.484502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd6db9-c54f-41c2-892b-904933b8b248 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:26.485855 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-07bd6db9-c54f-41c2-892b-904933b8b248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:26.543017 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd6db9-c54f-41c2-892b-904933b8b248 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:26.546818 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310cb12-e4e1-4d47-ad24-0bcf3d24419c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:26.547475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310cb12-e4e1-4d47-ad24-0bcf3d24419c service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:26.547788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310cb12-e4e1-4d47-ad24-0bcf3d24419c service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:26.548066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310cb12-e4e1-4d47-ad24-0bcf3d24419c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:26.549796 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a310cb12-e4e1-4d47-ad24-0bcf3d24419c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:26.608654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310cb12-e4e1-4d47-ad24-0bcf3d24419c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:26.610079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310cb12-e4e1-4d47-ad24-0bcf3d24419c service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:26.619618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1327/2658] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:02:34.093984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:34.229174 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1d070-2598-442b-aab8-7a482607a4b8 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:34.292404 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1d070-2598-442b-aab8-7a482607a4b8 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:34.297034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e53c8-dccb-44a8-a004-03f1a70fc375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:34.297597 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e53c8-dccb-44a8-a004-03f1a70fc375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:34.297873 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e53c8-dccb-44a8-a004-03f1a70fc375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:34.298142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e53c8-dccb-44a8-a004-03f1a70fc375 admin admin] PATH_INFO: `/v3/users/efebb7fe59f448fc8a827ac83877520a` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:34.299278 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e53c8-dccb-44a8-a004-03f1a70fc375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efebb7fe59f448fc8a827ac83877520a)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:34.301225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e53c8-dccb-44a8-a004-03f1a70fc375 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:34.465079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d68e53c8-dccb-44a8-a004-03f1a70fc375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efebb7fe59f448fc8a827ac83877520a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:34.477173 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d68e53c8-dccb-44a8-a004-03f1a70fc375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efebb7fe59f448fc8a827ac83877520a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:34.484092 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d68e53c8-dccb-44a8-a004-03f1a70fc375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efebb7fe59f448fc8a827ac83877520a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:34.573299 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d68e53c8-dccb-44a8-a004-03f1a70fc375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efebb7fe59f448fc8a827ac83877520a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:34.573872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-d68e53c8-dccb-44a8-a004-03f1a70fc375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efebb7fe59f448fc8a827ac83877520a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:34.580267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1332/2659] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:02:34 2022] DELETE /identity/v3/users/efebb7fe59f448fc8a827ac83877520a => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:34.584902 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12c61e-95ee-479e-b15d-c0d2d625017e None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:34.585351 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12c61e-95ee-479e-b15d-c0d2d625017e None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:34.585435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12c61e-95ee-479e-b15d-c0d2d625017e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:34.604733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-6b12c61e-95ee-479e-b15d-c0d2d625017e None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:34.606032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-6b12c61e-95ee-479e-b15d-c0d2d625017e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:34.667052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1328/2660] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:02:34.676797 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e53c8-dccb-44a8-a004-03f1a70fc375 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:34.678210 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d68e53c8-dccb-44a8-a004-03f1a70fc375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:34.735188 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e53c8-dccb-44a8-a004-03f1a70fc375 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:34.738857 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23589f72-5977-4442-8388-a518f0bcd705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:34.739497 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23589f72-5977-4442-8388-a518f0bcd705 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:34.739781 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23589f72-5977-4442-8388-a518f0bcd705 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:34.740057 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23589f72-5977-4442-8388-a518f0bcd705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:34.749831 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23589f72-5977-4442-8388-a518f0bcd705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:34.751311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23589f72-5977-4442-8388-a518f0bcd705 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:34.761518 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1333/2661] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:02:35.175502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12c61e-95ee-479e-b15d-c0d2d625017e None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:35.177317 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-6b12c61e-95ee-479e-b15d-c0d2d625017e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:35.195413 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12c61e-95ee-479e-b15d-c0d2d625017e None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:35.200387 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c182c0-9557-48ea-ab62-1b31fb86caa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:35.204794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c182c0-9557-48ea-ab62-1b31fb86caa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:35.204794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c182c0-9557-48ea-ab62-1b31fb86caa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:35.204794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c182c0-9557-48ea-ab62-1b31fb86caa6 admin admin] PATH_INFO: `/v3/projects/a82995e77f0e435c9d16d0d23affebb6` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:35.204794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c182c0-9557-48ea-ab62-1b31fb86caa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a82995e77f0e435c9d16d0d23affebb6)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:35.204794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c182c0-9557-48ea-ab62-1b31fb86caa6 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:35.342237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:35.343805 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f0c182c0-9557-48ea-ab62-1b31fb86caa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a82995e77f0e435c9d16d0d23affebb6', 'request_id': 'req-f0c182c0-9557-48ea-ab62-1b31fb86caa6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:35.349279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f0c182c0-9557-48ea-ab62-1b31fb86caa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a82995e77f0e435c9d16d0d23affebb6', 'request_id': 'req-f0c182c0-9557-48ea-ab62-1b31fb86caa6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:35.424378 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f0c182c0-9557-48ea-ab62-1b31fb86caa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a82995e77f0e435c9d16d0d23affebb6', 'request_id': 'req-f0c182c0-9557-48ea-ab62-1b31fb86caa6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:35.437028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f0c182c0-9557-48ea-ab62-1b31fb86caa6 admin admin] The token cache is being invalidate because project a82995e77f0e435c9d16d0d23affebb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:02:35.437327 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f0c182c0-9557-48ea-ab62-1b31fb86caa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:35.437987 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-f0c182c0-9557-48ea-ab62-1b31fb86caa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:35.439650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1329/2662] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:02:35 2022] DELETE /identity/v3/projects/a82995e77f0e435c9d16d0d23affebb6 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:02:35.444583 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcffe7-1386-4a6f-9d14-899976b9a5c1 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:35.445159 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcffe7-1386-4a6f-9d14-899976b9a5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:35.445248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcffe7-1386-4a6f-9d14-899976b9a5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:35.465733 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-1afcffe7-1386-4a6f-9d14-899976b9a5c1 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:35.466956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1afcffe7-1386-4a6f-9d14-899976b9a5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:35.485810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:35.528934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1334/2663] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:35.536940 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086d7ab-84d4-41b7-b732-631a6cdc9c5c None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:35.537185 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086d7ab-84d4-41b7-b732-631a6cdc9c5c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:35.537466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086d7ab-84d4-41b7-b732-631a6cdc9c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:35.558263 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-2086d7ab-84d4-41b7-b732-631a6cdc9c5c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:35.559657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2086d7ab-84d4-41b7-b732-631a6cdc9c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:35.568086 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1330/2664] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:02:35.572982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcffe7-1386-4a6f-9d14-899976b9a5c1 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:35.583656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcffe7-1386-4a6f-9d14-899976b9a5c1 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:35.589388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4ce4c-22ad-460e-9851-5a3461c5c0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:35.590006 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4ce4c-22ad-460e-9851-5a3461c5c0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:35.590331 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4ce4c-22ad-460e-9851-5a3461c5c0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:35.590602 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4ce4c-22ad-460e-9851-5a3461c5c0e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:35.591287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4ce4c-22ad-460e-9851-5a3461c5c0e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:02:35.591594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4ce4c-22ad-460e-9851-5a3461c5c0e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:35.593199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4ce4c-22ad-460e-9851-5a3461c5c0e1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:35.605300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1335/2665] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:02:35.609776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086d7ab-84d4-41b7-b732-631a6cdc9c5c None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:35.619349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086d7ab-84d4-41b7-b732-631a6cdc9c5c None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:35.624032 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82e48b-d773-423d-8f31-9167e9f3067f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:35.624629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82e48b-d773-423d-8f31-9167e9f3067f admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:35.624911 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82e48b-d773-423d-8f31-9167e9f3067f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:35.625185 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82e48b-d773-423d-8f31-9167e9f3067f admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:35.625923 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82e48b-d773-423d-8f31-9167e9f3067f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:35.627867 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82e48b-d773-423d-8f31-9167e9f3067f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:35.740026 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1331/2666] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:02:35 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:02:35.743986 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4ce4c-22ad-460e-9851-5a3461c5c0e1 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:35.754523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4ce4c-22ad-460e-9851-5a3461c5c0e1 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:35.812902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7748d-88ce-4d0d-b2ac-7d5c92278019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:35.813589 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7748d-88ce-4d0d-b2ac-7d5c92278019 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:35.813876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7748d-88ce-4d0d-b2ac-7d5c92278019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:35.814154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7748d-88ce-4d0d-b2ac-7d5c92278019 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:35.814803 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7748d-88ce-4d0d-b2ac-7d5c92278019 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:35.816766 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7748d-88ce-4d0d-b2ac-7d5c92278019 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:35.907293 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1336/2667] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:02:35 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:02:35.909461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82e48b-d773-423d-8f31-9167e9f3067f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:35.920342 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82e48b-d773-423d-8f31-9167e9f3067f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:35.924865 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0c9c3-f511-4a93-84ff-d0251b25f73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:35.925461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0c9c3-f511-4a93-84ff-d0251b25f73f admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:35.925738 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0c9c3-f511-4a93-84ff-d0251b25f73f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:35.926007 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0c9c3-f511-4a93-84ff-d0251b25f73f admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:35.926635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0c9c3-f511-4a93-84ff-d0251b25f73f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:35.928252 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0c9c3-f511-4a93-84ff-d0251b25f73f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:35.939019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1332/2668] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:02:35.943039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7748d-88ce-4d0d-b2ac-7d5c92278019 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:35.953990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7748d-88ce-4d0d-b2ac-7d5c92278019 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:35.961588 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e464575-dda5-434d-b99f-ea8055d1a9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:35.961763 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e464575-dda5-434d-b99f-ea8055d1a9b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:35.962037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e464575-dda5-434d-b99f-ea8055d1a9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:35.962456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e464575-dda5-434d-b99f-ea8055d1a9b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:35.963662 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e464575-dda5-434d-b99f-ea8055d1a9b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:35.965426 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e464575-dda5-434d-b99f-ea8055d1a9b2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:35.989979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0c9c3-f511-4a93-84ff-d0251b25f73f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:35.991137 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1337/2669] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:02:36.001585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0c9c3-f511-4a93-84ff-d0251b25f73f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:36.008174 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527d65f-a359-4eea-bdbe-7575c2362b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:36.008864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527d65f-a359-4eea-bdbe-7575c2362b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:36.009153 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527d65f-a359-4eea-bdbe-7575c2362b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:36.009599 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527d65f-a359-4eea-bdbe-7575c2362b79 admin admin] PATH_INFO: `/v3/projects/3275276850704a7f9390c94577e10428/users/826a360853ad42a4a794e5781c38e107/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:36.026981 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527d65f-a359-4eea-bdbe-7575c2362b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3275276850704a7f9390c94577e10428, user_id=826a360853ad42a4a794e5781c38e107, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:36.029669 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527d65f-a359-4eea-bdbe-7575c2362b79 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:36.103054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1333/2670] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:02:35 2022] PUT /identity/v3/projects/3275276850704a7f9390c94577e10428/users/826a360853ad42a4a794e5781c38e107/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:36.115071 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977e14e-c387-4f85-88a7-dc5d36a2c158 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:36.115395 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977e14e-c387-4f85-88a7-dc5d36a2c158 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:36.115694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977e14e-c387-4f85-88a7-dc5d36a2c158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:36.146215 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-7977e14e-c387-4f85-88a7-dc5d36a2c158 None None] MFA Rules not processed for user `826a360853ad42a4a794e5781c38e107`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:36.147494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7977e14e-c387-4f85-88a7-dc5d36a2c158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:36.217982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1338/2671] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:36.222686 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527d65f-a359-4eea-bdbe-7575c2362b79 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:36.300943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977e14e-c387-4f85-88a7-dc5d36a2c158 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:36.303364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7977e14e-c387-4f85-88a7-dc5d36a2c158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:36.319194 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527d65f-a359-4eea-bdbe-7575c2362b79 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:36.325325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ac3b7-a6e9-4fc5-8986-a4ace18ca716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:36.325962 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ac3b7-a6e9-4fc5-8986-a4ace18ca716 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:36.326243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ac3b7-a6e9-4fc5-8986-a4ace18ca716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:36.326522 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ac3b7-a6e9-4fc5-8986-a4ace18ca716 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:36.327159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ac3b7-a6e9-4fc5-8986-a4ace18ca716 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:36.329260 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ac3b7-a6e9-4fc5-8986-a4ace18ca716 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:36.331900 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:36.399105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977e14e-c387-4f85-88a7-dc5d36a2c158 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:36.406019 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460941d-811f-40c5-b6ad-3aa092a3ae71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:36.407012 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460941d-811f-40c5-b6ad-3aa092a3ae71 service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:36.407012 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460941d-811f-40c5-b6ad-3aa092a3ae71 service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:36.407340 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460941d-811f-40c5-b6ad-3aa092a3ae71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:36.409455 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2460941d-811f-40c5-b6ad-3aa092a3ae71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:36.423902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460941d-811f-40c5-b6ad-3aa092a3ae71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:36.425492 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460941d-811f-40c5-b6ad-3aa092a3ae71 service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:36.427066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1334/2672] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:02:36 2022] POST /identity/v3/projects => generated 383 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:02:36.431228 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ac3b7-a6e9-4fc5-8986-a4ace18ca716 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:36.441540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1339/2673] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:02:36.511363 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:36.516499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ac3b7-a6e9-4fc5-8986-a4ace18ca716 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:36.527523 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b1d4d-2245-45ea-96d7-5529a799e86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:36.528162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b1d4d-2245-45ea-96d7-5529a799e86e admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:36.528443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b1d4d-2245-45ea-96d7-5529a799e86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:36.528716 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b1d4d-2245-45ea-96d7-5529a799e86e admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:36.529360 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b1d4d-2245-45ea-96d7-5529a799e86e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:36.531377 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b1d4d-2245-45ea-96d7-5529a799e86e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:36.665568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1335/2674] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:02:36 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:02:36.669448 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460941d-811f-40c5-b6ad-3aa092a3ae71 service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:36.680446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460941d-811f-40c5-b6ad-3aa092a3ae71 service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:36.686037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c636d28-709c-4b76-aa43-a110d4d79ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:36.686742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c636d28-709c-4b76-aa43-a110d4d79ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:36.687111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c636d28-709c-4b76-aa43-a110d4d79ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:36.687491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c636d28-709c-4b76-aa43-a110d4d79ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:36.688220 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c636d28-709c-4b76-aa43-a110d4d79ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:36.689886 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c636d28-709c-4b76-aa43-a110d4d79ca1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:36.701370 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1340/2675] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:02:36.711961 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b1d4d-2245-45ea-96d7-5529a799e86e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:36.722559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b1d4d-2245-45ea-96d7-5529a799e86e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:36.727588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84b7a8-5df1-4250-87e3-0d129098301a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:36.728201 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84b7a8-5df1-4250-87e3-0d129098301a admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:36.728486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84b7a8-5df1-4250-87e3-0d129098301a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:36.728757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84b7a8-5df1-4250-87e3-0d129098301a admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:36.729403 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84b7a8-5df1-4250-87e3-0d129098301a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:36.731002 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84b7a8-5df1-4250-87e3-0d129098301a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:36.742292 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1336/2676] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:02:36.752381 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c636d28-709c-4b76-aa43-a110d4d79ca1 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:36.763067 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c636d28-709c-4b76-aa43-a110d4d79ca1 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:36.767847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b3b03-c563-41d3-86df-1c4ce28c22f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:36.768512 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b3b03-c563-41d3-86df-1c4ce28c22f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:36.768791 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b3b03-c563-41d3-86df-1c4ce28c22f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:36.769326 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b3b03-c563-41d3-86df-1c4ce28c22f3 admin admin] PATH_INFO: `/v3/projects/2df84086cdac4f2eb5ea7f5c079d7bd4/users/e4fe27911bec4037a18fa12491de4cfb/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:36.784874 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b3b03-c563-41d3-86df-1c4ce28c22f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2df84086cdac4f2eb5ea7f5c079d7bd4, user_id=e4fe27911bec4037a18fa12491de4cfb, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:36.787562 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b3b03-c563-41d3-86df-1c4ce28c22f3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:36.914644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1341/2677] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:02:36 2022] PUT /identity/v3/projects/2df84086cdac4f2eb5ea7f5c079d7bd4/users/e4fe27911bec4037a18fa12491de4cfb/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:36.927161 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de50ff-48c1-457e-b5f5-cc8f53434651 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:36.927658 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de50ff-48c1-457e-b5f5-cc8f53434651 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:36.928015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de50ff-48c1-457e-b5f5-cc8f53434651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:36.959077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-96de50ff-48c1-457e-b5f5-cc8f53434651 None None] MFA Rules not processed for user `e4fe27911bec4037a18fa12491de4cfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:36.960596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-96de50ff-48c1-457e-b5f5-cc8f53434651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:37.028706 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1337/2678] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:37.033280 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b3b03-c563-41d3-86df-1c4ce28c22f3 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:37.112618 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b3b03-c563-41d3-86df-1c4ce28c22f3 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:37.117893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0bb679-da74-4547-afcb-8c8a01b12965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:37.118836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bb679-da74-4547-afcb-8c8a01b12965 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:37.119417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bb679-da74-4547-afcb-8c8a01b12965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:37.120590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bb679-da74-4547-afcb-8c8a01b12965 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:37.121454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0bb679-da74-4547-afcb-8c8a01b12965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:37.123680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0bb679-da74-4547-afcb-8c8a01b12965 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:37.212231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1342/2679] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:02:37 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:02:37.213724 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de50ff-48c1-457e-b5f5-cc8f53434651 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:37.224949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de50ff-48c1-457e-b5f5-cc8f53434651 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:37.290168 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe4861-978b-408e-bea2-f5774ea1fc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:37.290689 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe4861-978b-408e-bea2-f5774ea1fc0f admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:37.290960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe4861-978b-408e-bea2-f5774ea1fc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:37.291853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe4861-978b-408e-bea2-f5774ea1fc0f admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:37.292492 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe4861-978b-408e-bea2-f5774ea1fc0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:37.294494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe4861-978b-408e-bea2-f5774ea1fc0f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:37.392721 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1338/2680] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:02:37 2022] POST /identity/v3/users => generated 299 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:02:37.394302 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0bb679-da74-4547-afcb-8c8a01b12965 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:37.405290 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0bb679-da74-4547-afcb-8c8a01b12965 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:37.412267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130512d-a0f5-405b-9bf6-f8e30a9058f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:37.412921 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130512d-a0f5-405b-9bf6-f8e30a9058f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:37.413200 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130512d-a0f5-405b-9bf6-f8e30a9058f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:37.413480 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130512d-a0f5-405b-9bf6-f8e30a9058f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:37.414142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130512d-a0f5-405b-9bf6-f8e30a9058f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:37.417214 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130512d-a0f5-405b-9bf6-f8e30a9058f8 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:37.428608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1343/2681] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:02:37.433365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe4861-978b-408e-bea2-f5774ea1fc0f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:37.444020 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe4861-978b-408e-bea2-f5774ea1fc0f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:37.449396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e85182-23c8-48c3-b834-9296fd879076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:37.450087 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e85182-23c8-48c3-b834-9296fd879076 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:37.450389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e85182-23c8-48c3-b834-9296fd879076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:37.450664 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e85182-23c8-48c3-b834-9296fd879076 admin admin] PATH_INFO: `/v3/projects/fa7da2ea35044bf988c98d69f0bbb520/users/4c754e795c4a47758544adad38fd763e/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:37.466383 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e85182-23c8-48c3-b834-9296fd879076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa7da2ea35044bf988c98d69f0bbb520, user_id=4c754e795c4a47758544adad38fd763e, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:37.469080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e85182-23c8-48c3-b834-9296fd879076 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:37.532344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1339/2682] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:02:37 2022] PUT /identity/v3/projects/fa7da2ea35044bf988c98d69f0bbb520/users/4c754e795c4a47758544adad38fd763e/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:37.536397 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130512d-a0f5-405b-9bf6-f8e30a9058f8 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:37.547799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130512d-a0f5-405b-9bf6-f8e30a9058f8 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:37.593502 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:37.617456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f268c1-0e49-44d8-bf4d-bf97cfef5fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:37.618173 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f268c1-0e49-44d8-bf4d-bf97cfef5fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:37.618531 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f268c1-0e49-44d8-bf4d-bf97cfef5fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:37.618867 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f268c1-0e49-44d8-bf4d-bf97cfef5fc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:37.619658 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f268c1-0e49-44d8-bf4d-bf97cfef5fc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:37.621339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f268c1-0e49-44d8-bf4d-bf97cfef5fc6 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:37.631654 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1344/2683] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:02:37.636257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e85182-23c8-48c3-b834-9296fd879076 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:37.651577 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e85182-23c8-48c3-b834-9296fd879076 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:37.656963 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03729bc6-71e1-477c-9be8-aa3f642ea2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:37.657132 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03729bc6-71e1-477c-9be8-aa3f642ea2e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:37.657331 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03729bc6-71e1-477c-9be8-aa3f642ea2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:37.657606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03729bc6-71e1-477c-9be8-aa3f642ea2e7 admin admin] PATH_INFO: `/v3/projects/fa7da2ea35044bf988c98d69f0bbb520/users/4c754e795c4a47758544adad38fd763e/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:37.659609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03729bc6-71e1-477c-9be8-aa3f642ea2e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa7da2ea35044bf988c98d69f0bbb520, user_id=4c754e795c4a47758544adad38fd763e, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:37.662282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03729bc6-71e1-477c-9be8-aa3f642ea2e7 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:37.674088 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1340/2684] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:02:37 2022] PUT /identity/v3/projects/fa7da2ea35044bf988c98d69f0bbb520/users/4c754e795c4a47758544adad38fd763e/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:02:37.684545 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037bde0-d5b8-49c0-a075-aa024c8ec351 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:37.685009 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037bde0-d5b8-49c0-a075-aa024c8ec351 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:37.685094 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037bde0-d5b8-49c0-a075-aa024c8ec351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:37.715805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-a037bde0-d5b8-49c0-a075-aa024c8ec351 None None] MFA Rules not processed for user `4c754e795c4a47758544adad38fd763e`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:37.717059 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a037bde0-d5b8-49c0-a075-aa024c8ec351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:37.791837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1345/2685] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:37.805886 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03729bc6-71e1-477c-9be8-aa3f642ea2e7 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:37.807753 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-03729bc6-71e1-477c-9be8-aa3f642ea2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:37.829370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:37.870807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03729bc6-71e1-477c-9be8-aa3f642ea2e7 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:37.875930 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e3499-d4cc-471d-996c-d29a42329cf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:37.876621 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e3499-d4cc-471d-996c-d29a42329cf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:37.877254 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e3499-d4cc-471d-996c-d29a42329cf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:37.877514 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e3499-d4cc-471d-996c-d29a42329cf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:37.935129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e3499-d4cc-471d-996c-d29a42329cf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:37.936709 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e3499-d4cc-471d-996c-d29a42329cf2 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:37.945832 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1341/2686] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:02:39.958049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037bde0-d5b8-49c0-a075-aa024c8ec351 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:39.959504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-a037bde0-d5b8-49c0-a075-aa024c8ec351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:39.973097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037bde0-d5b8-49c0-a075-aa024c8ec351 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:39.976806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a463470-c79c-436c-9e06-c139c703db7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:39.977415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a463470-c79c-436c-9e06-c139c703db7c service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:39.977701 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a463470-c79c-436c-9e06-c139c703db7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:39.977974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a463470-c79c-436c-9e06-c139c703db7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:40.035785 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a463470-c79c-436c-9e06-c139c703db7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:40.037305 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a463470-c79c-436c-9e06-c139c703db7c service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:40.047963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1346/2687] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:02:41.778061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e3499-d4cc-471d-996c-d29a42329cf2 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:41.786693 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e3499-d4cc-471d-996c-d29a42329cf2 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:41.790595 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b639c-60f9-41da-b551-d55ed974d9f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:41.791814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b639c-60f9-41da-b551-d55ed974d9f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:41.792095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b639c-60f9-41da-b551-d55ed974d9f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:41.792364 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b639c-60f9-41da-b551-d55ed974d9f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:41.801491 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b639c-60f9-41da-b551-d55ed974d9f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:41.802928 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b639c-60f9-41da-b551-d55ed974d9f2 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:41.814016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1342/2688] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:02:42.304196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018522b-d3d8-44c3-bbd3-338a693ac483 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:42.304725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018522b-d3d8-44c3-bbd3-338a693ac483 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:42.304725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018522b-d3d8-44c3-bbd3-338a693ac483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:42.324911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-6018522b-d3d8-44c3-bbd3-338a693ac483 None None] MFA Rules not processed for user `7483b0e52765411d8444dfdd46549c5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:42.326573 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6018522b-d3d8-44c3-bbd3-338a693ac483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:42.382394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1347/2689] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:42.691405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b639c-60f9-41da-b551-d55ed974d9f2 service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:42.700473 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-da4b639c-60f9-41da-b551-d55ed974d9f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:42.749935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b639c-60f9-41da-b551-d55ed974d9f2 service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:42.753516 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26303d25-2330-4626-8f12-640943d350de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:42.754120 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26303d25-2330-4626-8f12-640943d350de service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:42.754401 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26303d25-2330-4626-8f12-640943d350de service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:42.754673 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26303d25-2330-4626-8f12-640943d350de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:42.756422 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-26303d25-2330-4626-8f12-640943d350de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:42.766567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26303d25-2330-4626-8f12-640943d350de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:42.768005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26303d25-2330-4626-8f12-640943d350de service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:42.776370 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1343/2690] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:02:46.600106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018522b-d3d8-44c3-bbd3-338a693ac483 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:46.609427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=100 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:46.666957 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018522b-d3d8-44c3-bbd3-338a693ac483 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:46.671782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f995f-bb07-46a8-9fe8-999befb18c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:46.672421 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f995f-bb07-46a8-9fe8-999befb18c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:46.672704 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f995f-bb07-46a8-9fe8-999befb18c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:46.672985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f995f-bb07-46a8-9fe8-999befb18c66 admin admin] PATH_INFO: `/v3/users/826a360853ad42a4a794e5781c38e107` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:46.674118 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f995f-bb07-46a8-9fe8-999befb18c66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=826a360853ad42a4a794e5781c38e107)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:46.676111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f995f-bb07-46a8-9fe8-999befb18c66 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:46.903421 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2f1f995f-bb07-46a8-9fe8-999befb18c66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '826a360853ad42a4a794e5781c38e107'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:46.919851 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2f1f995f-bb07-46a8-9fe8-999befb18c66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '826a360853ad42a4a794e5781c38e107'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:46.928296 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2f1f995f-bb07-46a8-9fe8-999befb18c66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '826a360853ad42a4a794e5781c38e107'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:47.023871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2f1f995f-bb07-46a8-9fe8-999befb18c66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '826a360853ad42a4a794e5781c38e107'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:47.024510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-2f1f995f-bb07-46a8-9fe8-999befb18c66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '826a360853ad42a4a794e5781c38e107'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:47.034504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1348/2691] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:02:46 2022] DELETE /identity/v3/users/826a360853ad42a4a794e5781c38e107 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:47.036393 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ecc0a-0e72-447f-9f89-ec492707c74c None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:47.036856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ecc0a-0e72-447f-9f89-ec492707c74c None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:47.037122 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ecc0a-0e72-447f-9f89-ec492707c74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:47.056558 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-371ecc0a-0e72-447f-9f89-ec492707c74c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:47.057780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-371ecc0a-0e72-447f-9f89-ec492707c74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:47.125995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1344/2692] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:47.138257 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f995f-bb07-46a8-9fe8-999befb18c66 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:47.139759 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2f1f995f-bb07-46a8-9fe8-999befb18c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:47.205849 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f995f-bb07-46a8-9fe8-999befb18c66 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:47.209525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d0d00-5326-48f2-9cc9-3f3c4200406b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:47.210134 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d0d00-5326-48f2-9cc9-3f3c4200406b service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:47.210486 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d0d00-5326-48f2-9cc9-3f3c4200406b service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:47.210761 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d0d00-5326-48f2-9cc9-3f3c4200406b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:47.230722 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d0d00-5326-48f2-9cc9-3f3c4200406b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:47.235051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d0d00-5326-48f2-9cc9-3f3c4200406b service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:47.242416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1349/2693] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:02:47.825466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ecc0a-0e72-447f-9f89-ec492707c74c None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:47.827160 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-371ecc0a-0e72-447f-9f89-ec492707c74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:47.835946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=100 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:47.844406 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ecc0a-0e72-447f-9f89-ec492707c74c None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:47.850295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e50d5-9381-4a83-a912-3366d9eae4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:47.851038 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e50d5-9381-4a83-a912-3366d9eae4c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:47.851372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e50d5-9381-4a83-a912-3366d9eae4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:47.851692 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e50d5-9381-4a83-a912-3366d9eae4c5 admin admin] PATH_INFO: `/v3/projects/3275276850704a7f9390c94577e10428` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:47.852857 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e50d5-9381-4a83-a912-3366d9eae4c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3275276850704a7f9390c94577e10428)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:47.855408 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e50d5-9381-4a83-a912-3366d9eae4c5 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:47.979260 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-246e50d5-9381-4a83-a912-3366d9eae4c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3275276850704a7f9390c94577e10428', 'request_id': 'req-246e50d5-9381-4a83-a912-3366d9eae4c5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:47.985534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-246e50d5-9381-4a83-a912-3366d9eae4c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3275276850704a7f9390c94577e10428', 'request_id': 'req-246e50d5-9381-4a83-a912-3366d9eae4c5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:48.079137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-246e50d5-9381-4a83-a912-3366d9eae4c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3275276850704a7f9390c94577e10428', 'request_id': 'req-246e50d5-9381-4a83-a912-3366d9eae4c5'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:48.092373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-246e50d5-9381-4a83-a912-3366d9eae4c5 admin admin] The token cache is being invalidate because project 3275276850704a7f9390c94577e10428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:02:48.092791 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-246e50d5-9381-4a83-a912-3366d9eae4c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:48.093570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-246e50d5-9381-4a83-a912-3366d9eae4c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:48.095102 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1345/2694] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:02:47 2022] DELETE /identity/v3/projects/3275276850704a7f9390c94577e10428 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:48.099336 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d0d00-5326-48f2-9cc9-3f3c4200406b service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:48.117404 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d0d00-5326-48f2-9cc9-3f3c4200406b service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:48.146257 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:48.180009 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba89645-0296-466f-836c-4dc89b173666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:48.180661 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba89645-0296-466f-836c-4dc89b173666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:48.180943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba89645-0296-466f-836c-4dc89b173666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:48.181217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba89645-0296-466f-836c-4dc89b173666 admin admin] PATH_INFO: `/v3/users/e4fe27911bec4037a18fa12491de4cfb` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:48.182491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba89645-0296-466f-836c-4dc89b173666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4fe27911bec4037a18fa12491de4cfb)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:48.184440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba89645-0296-466f-836c-4dc89b173666 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:48.374628 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8ba89645-0296-466f-836c-4dc89b173666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4fe27911bec4037a18fa12491de4cfb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:48.387787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8ba89645-0296-466f-836c-4dc89b173666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4fe27911bec4037a18fa12491de4cfb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:48.395594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8ba89645-0296-466f-836c-4dc89b173666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4fe27911bec4037a18fa12491de4cfb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:48.457810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8ba89645-0296-466f-836c-4dc89b173666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4fe27911bec4037a18fa12491de4cfb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:48.458371 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8ba89645-0296-466f-836c-4dc89b173666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4fe27911bec4037a18fa12491de4cfb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:48.465867 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1350/2695] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:02:48 2022] DELETE /identity/v3/users/e4fe27911bec4037a18fa12491de4cfb => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:02:48.846424 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e50d5-9381-4a83-a912-3366d9eae4c5 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:48.847924 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-246e50d5-9381-4a83-a912-3366d9eae4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:48.860862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e50d5-9381-4a83-a912-3366d9eae4c5 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:48.918724 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb401113-1ad1-4813-a814-1c6f94e8ccc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:48.919314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb401113-1ad1-4813-a814-1c6f94e8ccc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:48.919597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb401113-1ad1-4813-a814-1c6f94e8ccc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:48.919873 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb401113-1ad1-4813-a814-1c6f94e8ccc2 admin admin] PATH_INFO: `/v3/projects/2df84086cdac4f2eb5ea7f5c079d7bd4` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:48.921003 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb401113-1ad1-4813-a814-1c6f94e8ccc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2df84086cdac4f2eb5ea7f5c079d7bd4)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:48.922908 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb401113-1ad1-4813-a814-1c6f94e8ccc2 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:49.026666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-cb401113-1ad1-4813-a814-1c6f94e8ccc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2df84086cdac4f2eb5ea7f5c079d7bd4', 'request_id': 'req-cb401113-1ad1-4813-a814-1c6f94e8ccc2'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:49.032711 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-cb401113-1ad1-4813-a814-1c6f94e8ccc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2df84086cdac4f2eb5ea7f5c079d7bd4', 'request_id': 'req-cb401113-1ad1-4813-a814-1c6f94e8ccc2'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:49.097799 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-cb401113-1ad1-4813-a814-1c6f94e8ccc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2df84086cdac4f2eb5ea7f5c079d7bd4', 'request_id': 'req-cb401113-1ad1-4813-a814-1c6f94e8ccc2'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:49.111827 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-cb401113-1ad1-4813-a814-1c6f94e8ccc2 admin admin] The token cache is being invalidate because project 2df84086cdac4f2eb5ea7f5c079d7bd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:02:49.112123 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-cb401113-1ad1-4813-a814-1c6f94e8ccc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:49.113000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-cb401113-1ad1-4813-a814-1c6f94e8ccc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:49.115034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1346/2696] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:02:48 2022] DELETE /identity/v3/projects/2df84086cdac4f2eb5ea7f5c079d7bd4 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:49.119824 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba89645-0296-466f-836c-4dc89b173666 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:49.121297 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8ba89645-0296-466f-836c-4dc89b173666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:49.135121 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba89645-0296-466f-836c-4dc89b173666 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:49.201298 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd96dfc-cd72-4d73-bc0a-8ed352eadfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:49.201861 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd96dfc-cd72-4d73-bc0a-8ed352eadfbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:49.202144 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd96dfc-cd72-4d73-bc0a-8ed352eadfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:49.202419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd96dfc-cd72-4d73-bc0a-8ed352eadfbb admin admin] PATH_INFO: `/v3/users/4c754e795c4a47758544adad38fd763e` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:49.203629 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd96dfc-cd72-4d73-bc0a-8ed352eadfbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c754e795c4a47758544adad38fd763e)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:49.205550 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd96dfc-cd72-4d73-bc0a-8ed352eadfbb admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:49.386431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0bd96dfc-cd72-4d73-bc0a-8ed352eadfbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c754e795c4a47758544adad38fd763e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:49.388559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:49.401333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0bd96dfc-cd72-4d73-bc0a-8ed352eadfbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c754e795c4a47758544adad38fd763e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:49.408234 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0bd96dfc-cd72-4d73-bc0a-8ed352eadfbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c754e795c4a47758544adad38fd763e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:49.480803 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0bd96dfc-cd72-4d73-bc0a-8ed352eadfbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c754e795c4a47758544adad38fd763e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:49.481329 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-0bd96dfc-cd72-4d73-bc0a-8ed352eadfbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c754e795c4a47758544adad38fd763e'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:49.487640 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1351/2697] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:02:49 2022] DELETE /identity/v3/users/4c754e795c4a47758544adad38fd763e => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:49.864059 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb401113-1ad1-4813-a814-1c6f94e8ccc2 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:49.866015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-cb401113-1ad1-4813-a814-1c6f94e8ccc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:49.883453 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb401113-1ad1-4813-a814-1c6f94e8ccc2 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:49.899833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:49.945094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8de005-0987-4b34-af4a-99edaa8eaa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:49.945730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8de005-0987-4b34-af4a-99edaa8eaa1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:49.946019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8de005-0987-4b34-af4a-99edaa8eaa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:49.946298 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8de005-0987-4b34-af4a-99edaa8eaa1e admin admin] PATH_INFO: `/v3/projects/fa7da2ea35044bf988c98d69f0bbb520` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:49.948019 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8de005-0987-4b34-af4a-99edaa8eaa1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa7da2ea35044bf988c98d69f0bbb520)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:49.950724 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8de005-0987-4b34-af4a-99edaa8eaa1e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:50.053626 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1b8de005-0987-4b34-af4a-99edaa8eaa1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa7da2ea35044bf988c98d69f0bbb520', 'request_id': 'req-1b8de005-0987-4b34-af4a-99edaa8eaa1e'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:50.060321 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1b8de005-0987-4b34-af4a-99edaa8eaa1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa7da2ea35044bf988c98d69f0bbb520', 'request_id': 'req-1b8de005-0987-4b34-af4a-99edaa8eaa1e'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:50.126649 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1b8de005-0987-4b34-af4a-99edaa8eaa1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa7da2ea35044bf988c98d69f0bbb520', 'request_id': 'req-1b8de005-0987-4b34-af4a-99edaa8eaa1e'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:50.140466 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1b8de005-0987-4b34-af4a-99edaa8eaa1e admin admin] The token cache is being invalidate because project fa7da2ea35044bf988c98d69f0bbb520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:02:50.140760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1b8de005-0987-4b34-af4a-99edaa8eaa1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:50.141421 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1b8de005-0987-4b34-af4a-99edaa8eaa1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:02:50.142954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1347/2698] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:02:49 2022] DELETE /identity/v3/projects/fa7da2ea35044bf988c98d69f0bbb520 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:50.148388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8f786-003d-4298-be07-750c604a933b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:50.148881 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8f786-003d-4298-be07-750c604a933b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:50.148972 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8f786-003d-4298-be07-750c604a933b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:50.169768 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-17f8f786-003d-4298-be07-750c604a933b None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:50.171228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-17f8f786-003d-4298-be07-750c604a933b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:50.234790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1352/2699] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:50.244437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730fad7-f9ff-40f8-9b38-0f7c7fec8ff1 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:50.244737 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730fad7-f9ff-40f8-9b38-0f7c7fec8ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:50.245015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730fad7-f9ff-40f8-9b38-0f7c7fec8ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:50.266243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-4730fad7-f9ff-40f8-9b38-0f7c7fec8ff1 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:50.267516 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4730fad7-f9ff-40f8-9b38-0f7c7fec8ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:50.275903 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1348/2700] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:50.280228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8f786-003d-4298-be07-750c604a933b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:50.290447 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8f786-003d-4298-be07-750c604a933b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:50.295464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ac9a1-79ba-48e1-97e9-e939abfdc769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:50.296084 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ac9a1-79ba-48e1-97e9-e939abfdc769 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:50.296396 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ac9a1-79ba-48e1-97e9-e939abfdc769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:50.296672 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ac9a1-79ba-48e1-97e9-e939abfdc769 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:50.297378 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ac9a1-79ba-48e1-97e9-e939abfdc769 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:02:50.297632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ac9a1-79ba-48e1-97e9-e939abfdc769 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:50.299255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ac9a1-79ba-48e1-97e9-e939abfdc769 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:50.315871 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1353/2701] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:02:50.317066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730fad7-f9ff-40f8-9b38-0f7c7fec8ff1 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:50.327020 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730fad7-f9ff-40f8-9b38-0f7c7fec8ff1 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:50.331776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72993d-96a3-4304-ab69-6d5e4de384cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:50.332406 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72993d-96a3-4304-ab69-6d5e4de384cb admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:50.332688 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72993d-96a3-4304-ab69-6d5e4de384cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:50.332959 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72993d-96a3-4304-ab69-6d5e4de384cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:50.333590 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72993d-96a3-4304-ab69-6d5e4de384cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:50.335526 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72993d-96a3-4304-ab69-6d5e4de384cb admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:50.402773 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1349/2702] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:02:50 2022] POST /identity/v3/projects => generated 369 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:02:50.406840 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ac9a1-79ba-48e1-97e9-e939abfdc769 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:50.416981 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ac9a1-79ba-48e1-97e9-e939abfdc769 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:50.472717 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac173e-883e-4eb8-8e97-9cae6a5cf45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:50.473319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac173e-883e-4eb8-8e97-9cae6a5cf45d admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:50.473601 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac173e-883e-4eb8-8e97-9cae6a5cf45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:50.473876 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac173e-883e-4eb8-8e97-9cae6a5cf45d admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:50.474507 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac173e-883e-4eb8-8e97-9cae6a5cf45d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:50.476481 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac173e-883e-4eb8-8e97-9cae6a5cf45d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:50.561412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1354/2703] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:02:50 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:02:50.565749 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72993d-96a3-4304-ab69-6d5e4de384cb admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:50.576359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72993d-96a3-4304-ab69-6d5e4de384cb admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:50.581052 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4615c-6895-48ad-966f-41fef0b56baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:50.581648 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4615c-6895-48ad-966f-41fef0b56baa admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:50.581933 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4615c-6895-48ad-966f-41fef0b56baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:50.582254 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4615c-6895-48ad-966f-41fef0b56baa admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:50.582892 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4615c-6895-48ad-966f-41fef0b56baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:50.584507 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4615c-6895-48ad-966f-41fef0b56baa admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:50.594450 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1350/2704] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:02:50.598601 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac173e-883e-4eb8-8e97-9cae6a5cf45d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:50.609165 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac173e-883e-4eb8-8e97-9cae6a5cf45d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:50.613748 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dcdbbe-833d-4f55-8e4d-f8f94d203dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:50.614335 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcdbbe-833d-4f55-8e4d-f8f94d203dda admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:50.614611 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcdbbe-833d-4f55-8e4d-f8f94d203dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:50.614883 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcdbbe-833d-4f55-8e4d-f8f94d203dda admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:50.615694 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dcdbbe-833d-4f55-8e4d-f8f94d203dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:50.617329 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dcdbbe-833d-4f55-8e4d-f8f94d203dda admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:50.628523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1355/2705] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:02:50.632958 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4615c-6895-48ad-966f-41fef0b56baa admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:50.643085 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4615c-6895-48ad-966f-41fef0b56baa admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:50.648273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc47e98-75e2-46db-b4dc-6d6be9f51292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:50.648926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc47e98-75e2-46db-b4dc-6d6be9f51292 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:50.649207 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc47e98-75e2-46db-b4dc-6d6be9f51292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:50.649482 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc47e98-75e2-46db-b4dc-6d6be9f51292 admin admin] PATH_INFO: `/v3/projects/6fa4dc2f16ff4ae8903f936984f9ba53/users/585e2706642a48c9ac002d10e86de96e/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:50.663966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc47e98-75e2-46db-b4dc-6d6be9f51292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fa4dc2f16ff4ae8903f936984f9ba53, user_id=585e2706642a48c9ac002d10e86de96e, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:50.667112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc47e98-75e2-46db-b4dc-6d6be9f51292 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:50.736656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1351/2706] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:02:50 2022] PUT /identity/v3/projects/6fa4dc2f16ff4ae8903f936984f9ba53/users/585e2706642a48c9ac002d10e86de96e/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:02:50.747570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5217f36-f788-47f5-9052-1bb4ad79af70 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:50.747860 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5217f36-f788-47f5-9052-1bb4ad79af70 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:50.748135 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5217f36-f788-47f5-9052-1bb4ad79af70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:50.776025 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e5217f36-f788-47f5-9052-1bb4ad79af70 None None] MFA Rules not processed for user `585e2706642a48c9ac002d10e86de96e`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:02:50.777285 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e5217f36-f788-47f5-9052-1bb4ad79af70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:50.797525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:50.841738 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1356/2707] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:02:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:02:50.853190 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc47e98-75e2-46db-b4dc-6d6be9f51292 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:02:50.855530 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5cc47e98-75e2-46db-b4dc-6d6be9f51292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:02:50.901715 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:02:50.913318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc47e98-75e2-46db-b4dc-6d6be9f51292 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:02:50.917024 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5cb58-3a32-4a83-8389-4a0b16ac98c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558e3b6dd9754c57ad028aa431a68b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:02:50.917630 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5cb58-3a32-4a83-8389-4a0b16ac98c2 service glance] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:02:50.917905 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5cb58-3a32-4a83-8389-4a0b16ac98c2 service glance] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:02:50.918176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5cb58-3a32-4a83-8389-4a0b16ac98c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:02:50.927998 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5cb58-3a32-4a83-8389-4a0b16ac98c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:02:50.929388 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5cb58-3a32-4a83-8389-4a0b16ac98c2 service glance] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:02:50.938957 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1352/2708] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:02:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:03:00.886513 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=17 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:00.966337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=7 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:02.955098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5217f36-f788-47f5-9052-1bb4ad79af70 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:02.956565 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e5217f36-f788-47f5-9052-1bb4ad79af70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:03.013867 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5217f36-f788-47f5-9052-1bb4ad79af70 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:03.017493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23a302-f386-43a6-b73f-52ed1b2548c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:03.018368 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23a302-f386-43a6-b73f-52ed1b2548c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:03.018902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23a302-f386-43a6-b73f-52ed1b2548c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:03.019199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23a302-f386-43a6-b73f-52ed1b2548c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:03.020953 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ea23a302-f386-43a6-b73f-52ed1b2548c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:03.084255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23a302-f386-43a6-b73f-52ed1b2548c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:03.085692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23a302-f386-43a6-b73f-52ed1b2548c2 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:03.095600 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1357/2709] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:03:06.604008 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5cb58-3a32-4a83-8389-4a0b16ac98c2 service glance] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:06.606329 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a3c5cb58-3a32-4a83-8389-4a0b16ac98c2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:06.686497 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5cb58-3a32-4a83-8389-4a0b16ac98c2 service glance] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:06.691839 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db8af7-df03-4607-a827-2fbaf9761a85 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:06.692451 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db8af7-df03-4607-a827-2fbaf9761a85 service placement] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:06.692731 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db8af7-df03-4607-a827-2fbaf9761a85 service placement] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:06.693006 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db8af7-df03-4607-a827-2fbaf9761a85 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:06.694774 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e9db8af7-df03-4607-a827-2fbaf9761a85 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:06.707577 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db8af7-df03-4607-a827-2fbaf9761a85 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:06.709184 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db8af7-df03-4607-a827-2fbaf9761a85 service placement] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:06.719337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1353/2710] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:03:06.738043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23a302-f386-43a6-b73f-52ed1b2548c2 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:06.739611 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ea23a302-f386-43a6-b73f-52ed1b2548c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:06.753729 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23a302-f386-43a6-b73f-52ed1b2548c2 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:06.759332 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2072d45-b2b9-4633-a10c-0a20c03dce6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:06.760394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2072d45-b2b9-4633-a10c-0a20c03dce6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:06.760628 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2072d45-b2b9-4633-a10c-0a20c03dce6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:06.760902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2072d45-b2b9-4633-a10c-0a20c03dce6f admin admin] PATH_INFO: `/v3/users/5a8fb193bdd1448eab774bd1eef437c9` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:06.762085 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2072d45-b2b9-4633-a10c-0a20c03dce6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a8fb193bdd1448eab774bd1eef437c9)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:06.764657 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2072d45-b2b9-4633-a10c-0a20c03dce6f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:06.957648 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a2072d45-b2b9-4633-a10c-0a20c03dce6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a8fb193bdd1448eab774bd1eef437c9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:06.970043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a2072d45-b2b9-4633-a10c-0a20c03dce6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a8fb193bdd1448eab774bd1eef437c9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:06.977160 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a2072d45-b2b9-4633-a10c-0a20c03dce6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a8fb193bdd1448eab774bd1eef437c9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:07.033457 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a2072d45-b2b9-4633-a10c-0a20c03dce6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a8fb193bdd1448eab774bd1eef437c9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:07.034111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-a2072d45-b2b9-4633-a10c-0a20c03dce6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a8fb193bdd1448eab774bd1eef437c9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:07.041103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1358/2711] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:03:06 2022] DELETE /identity/v3/users/5a8fb193bdd1448eab774bd1eef437c9 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:03:07.318837 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db8af7-df03-4607-a827-2fbaf9761a85 service placement] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:07.329699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db8af7-df03-4607-a827-2fbaf9761a85 service placement] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:07.410854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb8358-d804-4eab-b9bb-638105665187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:07.411587 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb8358-d804-4eab-b9bb-638105665187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:07.411800 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb8358-d804-4eab-b9bb-638105665187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:07.412068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb8358-d804-4eab-b9bb-638105665187 admin admin] PATH_INFO: `/v3/projects/0ddce6e2bc7841d495afa07655d7d25f` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:07.421596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb8358-d804-4eab-b9bb-638105665187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ddce6e2bc7841d495afa07655d7d25f)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:07.421596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb8358-d804-4eab-b9bb-638105665187 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:07.525268 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-28bb8358-d804-4eab-b9bb-638105665187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ddce6e2bc7841d495afa07655d7d25f', 'request_id': 'req-28bb8358-d804-4eab-b9bb-638105665187'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:07.530984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-28bb8358-d804-4eab-b9bb-638105665187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ddce6e2bc7841d495afa07655d7d25f', 'request_id': 'req-28bb8358-d804-4eab-b9bb-638105665187'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:07.588847 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-28bb8358-d804-4eab-b9bb-638105665187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ddce6e2bc7841d495afa07655d7d25f', 'request_id': 'req-28bb8358-d804-4eab-b9bb-638105665187'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:07.601368 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-28bb8358-d804-4eab-b9bb-638105665187 admin admin] The token cache is being invalidate because project 0ddce6e2bc7841d495afa07655d7d25f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:03:07.604002 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-28bb8358-d804-4eab-b9bb-638105665187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:07.604002 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-28bb8358-d804-4eab-b9bb-638105665187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:07.604002 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1354/2712] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:03:07 2022] DELETE /identity/v3/projects/0ddce6e2bc7841d495afa07655d7d25f => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:07.609185 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f06a0-5309-4588-ba02-1292f0f1ba62 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:07.609476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f06a0-5309-4588-ba02-1292f0f1ba62 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:07.609754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f06a0-5309-4588-ba02-1292f0f1ba62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:07.633927 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-628f06a0-5309-4588-ba02-1292f0f1ba62 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:07.635196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-628f06a0-5309-4588-ba02-1292f0f1ba62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:07.653916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:07.705924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1359/2713] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:07.716857 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc670a0-006e-4538-a469-64d3c631d866 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:07.718309 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc670a0-006e-4538-a469-64d3c631d866 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:07.718687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc670a0-006e-4538-a469-64d3c631d866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:07.740198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-1cc670a0-006e-4538-a469-64d3c631d866 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:07.741444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1cc670a0-006e-4538-a469-64d3c631d866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:07.750445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1355/2714] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:07.754973 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f06a0-5309-4588-ba02-1292f0f1ba62 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:07.765322 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f06a0-5309-4588-ba02-1292f0f1ba62 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:07.770278 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3bbab-8b7c-44d3-a144-4ed43cdbbb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:07.770917 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3bbab-8b7c-44d3-a144-4ed43cdbbb4d admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:07.771231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3bbab-8b7c-44d3-a144-4ed43cdbbb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:07.771510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3bbab-8b7c-44d3-a144-4ed43cdbbb4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:07.772158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3bbab-8b7c-44d3-a144-4ed43cdbbb4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:03:07.772462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3bbab-8b7c-44d3-a144-4ed43cdbbb4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:07.774049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3bbab-8b7c-44d3-a144-4ed43cdbbb4d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:07.786295 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1360/2715] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:03:07.790691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc670a0-006e-4538-a469-64d3c631d866 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:07.800528 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc670a0-006e-4538-a469-64d3c631d866 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:07.805247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfc5ff-1c56-404d-ac3f-6d161a9c232f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:07.805819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfc5ff-1c56-404d-ac3f-6d161a9c232f admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:07.806097 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfc5ff-1c56-404d-ac3f-6d161a9c232f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:07.806372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfc5ff-1c56-404d-ac3f-6d161a9c232f admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:07.807035 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfc5ff-1c56-404d-ac3f-6d161a9c232f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:07.809639 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfc5ff-1c56-404d-ac3f-6d161a9c232f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:07.822180 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:07.909108 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1356/2716] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:03:07 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:07.913229 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3bbab-8b7c-44d3-a144-4ed43cdbbb4d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:07.924050 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3bbab-8b7c-44d3-a144-4ed43cdbbb4d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:07.980546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a982569-46ac-4fe4-b663-639b4036ca30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:07.981732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a982569-46ac-4fe4-b663-639b4036ca30 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:07.982020 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a982569-46ac-4fe4-b663-639b4036ca30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:07.982292 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a982569-46ac-4fe4-b663-639b4036ca30 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:07.982924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a982569-46ac-4fe4-b663-639b4036ca30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:07.984898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a982569-46ac-4fe4-b663-639b4036ca30 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:08.087479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1361/2717] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:03:07 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:08.092063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfc5ff-1c56-404d-ac3f-6d161a9c232f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:08.102657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfc5ff-1c56-404d-ac3f-6d161a9c232f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:08.108001 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194091b-3fbd-4e91-888b-5a4f62aac1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:08.108606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194091b-3fbd-4e91-888b-5a4f62aac1ac admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:08.108912 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194091b-3fbd-4e91-888b-5a4f62aac1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:08.109186 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194091b-3fbd-4e91-888b-5a4f62aac1ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:08.109876 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194091b-3fbd-4e91-888b-5a4f62aac1ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:08.111457 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194091b-3fbd-4e91-888b-5a4f62aac1ac admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:08.123016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1357/2718] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:08.127047 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a982569-46ac-4fe4-b663-639b4036ca30 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:08.137524 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a982569-46ac-4fe4-b663-639b4036ca30 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:08.142300 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcbf45-dc8a-4632-8291-eca84fdc7f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:08.142943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcbf45-dc8a-4632-8291-eca84fdc7f8e admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:08.143482 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcbf45-dc8a-4632-8291-eca84fdc7f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:08.143773 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcbf45-dc8a-4632-8291-eca84fdc7f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:08.144391 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebcbf45-dc8a-4632-8291-eca84fdc7f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:08.145979 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebcbf45-dc8a-4632-8291-eca84fdc7f8e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:08.158892 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1362/2719] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:08.163225 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194091b-3fbd-4e91-888b-5a4f62aac1ac admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:08.174207 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194091b-3fbd-4e91-888b-5a4f62aac1ac admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:08.179149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c462e-e36d-44f6-b77e-bcd0abb8591d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:08.179844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c462e-e36d-44f6-b77e-bcd0abb8591d admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:08.180125 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c462e-e36d-44f6-b77e-bcd0abb8591d admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:08.180402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c462e-e36d-44f6-b77e-bcd0abb8591d admin admin] PATH_INFO: `/v3/projects/9ec5468c0b96400a80d42cd122c95466/users/83065c4d26da46fe84de678cb277f3e6/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:08.195658 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c462e-e36d-44f6-b77e-bcd0abb8591d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec5468c0b96400a80d42cd122c95466, user_id=83065c4d26da46fe84de678cb277f3e6, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:08.198723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c462e-e36d-44f6-b77e-bcd0abb8591d admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:08.292136 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1358/2720] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:03:08 2022] PUT /identity/v3/projects/9ec5468c0b96400a80d42cd122c95466/users/83065c4d26da46fe84de678cb277f3e6/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:08.304109 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259ea0d-9aad-4d86-ab13-7bbf4d4579ea None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:08.304406 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259ea0d-9aad-4d86-ab13-7bbf4d4579ea None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:08.304678 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259ea0d-9aad-4d86-ab13-7bbf4d4579ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:08.332853 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-1259ea0d-9aad-4d86-ab13-7bbf4d4579ea None None] MFA Rules not processed for user `83065c4d26da46fe84de678cb277f3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:08.334113 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1259ea0d-9aad-4d86-ab13-7bbf4d4579ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:08.401484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1363/2721] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:08.407656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c462e-e36d-44f6-b77e-bcd0abb8591d admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:08.470722 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c462e-e36d-44f6-b77e-bcd0abb8591d admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:08.475419 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c06ab-88cf-4526-bddc-2851a82bd734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:08.476023 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c06ab-88cf-4526-bddc-2851a82bd734 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:08.476353 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c06ab-88cf-4526-bddc-2851a82bd734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:08.476578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c06ab-88cf-4526-bddc-2851a82bd734 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:08.477214 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c06ab-88cf-4526-bddc-2851a82bd734 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:08.479111 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c06ab-88cf-4526-bddc-2851a82bd734 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:08.572938 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1359/2722] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:03:08 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:03:08.577698 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259ea0d-9aad-4d86-ab13-7bbf4d4579ea None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:08.588494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259ea0d-9aad-4d86-ab13-7bbf4d4579ea None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:08.602613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:08.655904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3013d71-aeb8-4739-8d78-7ba0d6d34815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:08.656518 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3013d71-aeb8-4739-8d78-7ba0d6d34815 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:08.656802 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3013d71-aeb8-4739-8d78-7ba0d6d34815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:08.657090 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3013d71-aeb8-4739-8d78-7ba0d6d34815 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:08.657723 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3013d71-aeb8-4739-8d78-7ba0d6d34815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:08.659670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3013d71-aeb8-4739-8d78-7ba0d6d34815 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:08.815669 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1364/2723] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:03:08 2022] POST /identity/v3/users => generated 294 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:08.816934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c06ab-88cf-4526-bddc-2851a82bd734 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:08.828262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c06ab-88cf-4526-bddc-2851a82bd734 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:08.833379 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90cedbc-cf6b-4647-a362-71b5558d9171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:08.834064 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cedbc-cf6b-4647-a362-71b5558d9171 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:08.834498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cedbc-cf6b-4647-a362-71b5558d9171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:08.834817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cedbc-cf6b-4647-a362-71b5558d9171 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:08.835571 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90cedbc-cf6b-4647-a362-71b5558d9171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:08.837232 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90cedbc-cf6b-4647-a362-71b5558d9171 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:08.848710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1360/2724] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:03:08.852956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3013d71-aeb8-4739-8d78-7ba0d6d34815 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:08.864716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3013d71-aeb8-4739-8d78-7ba0d6d34815 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:08.869641 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc14650-a8f7-4d9d-a4b6-3b6b08629408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:08.870393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc14650-a8f7-4d9d-a4b6-3b6b08629408 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:08.870686 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc14650-a8f7-4d9d-a4b6-3b6b08629408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:08.870946 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc14650-a8f7-4d9d-a4b6-3b6b08629408 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:08.872422 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc14650-a8f7-4d9d-a4b6-3b6b08629408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:08.874142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc14650-a8f7-4d9d-a4b6-3b6b08629408 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:08.888393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1365/2725] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:08.889593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90cedbc-cf6b-4647-a362-71b5558d9171 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:08.900875 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90cedbc-cf6b-4647-a362-71b5558d9171 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:08.905549 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a9071-72a8-4eda-86e7-d81392322d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:08.906204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a9071-72a8-4eda-86e7-d81392322d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:08.906486 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a9071-72a8-4eda-86e7-d81392322d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:08.906766 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a9071-72a8-4eda-86e7-d81392322d1f admin admin] PATH_INFO: `/v3/projects/13003d22da7846e2acc01323cc30eca8/users/58c87c26d9904845bfed892b4706f62c/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:08.922066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a9071-72a8-4eda-86e7-d81392322d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=13003d22da7846e2acc01323cc30eca8, user_id=58c87c26d9904845bfed892b4706f62c, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:08.924811 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a9071-72a8-4eda-86e7-d81392322d1f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:08.995578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1361/2726] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:03:08 2022] PUT /identity/v3/projects/13003d22da7846e2acc01323cc30eca8/users/58c87c26d9904845bfed892b4706f62c/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:09.005996 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691e406-0456-41bd-9c89-1529b4a47c75 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:09.006281 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691e406-0456-41bd-9c89-1529b4a47c75 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:09.006608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691e406-0456-41bd-9c89-1529b4a47c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:09.036413 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-f691e406-0456-41bd-9c89-1529b4a47c75 None None] MFA Rules not processed for user `58c87c26d9904845bfed892b4706f62c`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:09.037752 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f691e406-0456-41bd-9c89-1529b4a47c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:09.103371 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1366/2727] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:09.108139 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a9071-72a8-4eda-86e7-d81392322d1f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:09.172604 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a9071-72a8-4eda-86e7-d81392322d1f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:09.177445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1423da-f77e-4a10-b848-d8cc441ef3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:09.178046 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1423da-f77e-4a10-b848-d8cc441ef3c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:09.178325 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1423da-f77e-4a10-b848-d8cc441ef3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:09.178597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1423da-f77e-4a10-b848-d8cc441ef3c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:09.179250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1423da-f77e-4a10-b848-d8cc441ef3c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:09.181128 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1423da-f77e-4a10-b848-d8cc441ef3c9 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:09.193068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:09.311982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1362/2728] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:03:09 2022] POST /identity/v3/projects => generated 385 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:09.326985 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691e406-0456-41bd-9c89-1529b4a47c75 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:09.338912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691e406-0456-41bd-9c89-1529b4a47c75 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:09.400724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c1893-6bbc-45a7-90ef-fbfd9060e3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:09.401349 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c1893-6bbc-45a7-90ef-fbfd9060e3be admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:09.401628 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c1893-6bbc-45a7-90ef-fbfd9060e3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:09.401904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c1893-6bbc-45a7-90ef-fbfd9060e3be admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:09.402549 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c1893-6bbc-45a7-90ef-fbfd9060e3be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:09.404779 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c1893-6bbc-45a7-90ef-fbfd9060e3be admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:09.531354 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1367/2729] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:03:09 2022] POST /identity/v3/users => generated 299 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:09.535115 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1423da-f77e-4a10-b848-d8cc441ef3c9 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:09.545786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1423da-f77e-4a10-b848-d8cc441ef3c9 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:09.550382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06904896-8d94-4282-ad14-71e59fa5782e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:09.551664 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06904896-8d94-4282-ad14-71e59fa5782e admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:09.551664 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06904896-8d94-4282-ad14-71e59fa5782e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:09.551664 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06904896-8d94-4282-ad14-71e59fa5782e admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:09.552185 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06904896-8d94-4282-ad14-71e59fa5782e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:09.553833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06904896-8d94-4282-ad14-71e59fa5782e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:09.565375 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1363/2730] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:09.569586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c1893-6bbc-45a7-90ef-fbfd9060e3be admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:09.580106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c1893-6bbc-45a7-90ef-fbfd9060e3be admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:09.584785 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc085c-35a2-4db5-b6da-dc3fdd3eda53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:09.585445 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc085c-35a2-4db5-b6da-dc3fdd3eda53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:09.585727 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc085c-35a2-4db5-b6da-dc3fdd3eda53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:09.586004 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc085c-35a2-4db5-b6da-dc3fdd3eda53 admin admin] PATH_INFO: `/v3/projects/8e718c511df34177bd5a8b58c8b0dd0c/users/893540a1e1fa44b990c529363b994fce/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:09.600042 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:09.602963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc085c-35a2-4db5-b6da-dc3fdd3eda53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e718c511df34177bd5a8b58c8b0dd0c, user_id=893540a1e1fa44b990c529363b994fce, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:09.605744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc085c-35a2-4db5-b6da-dc3fdd3eda53 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:09.709247 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1368/2731] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:03:09 2022] PUT /identity/v3/projects/8e718c511df34177bd5a8b58c8b0dd0c/users/893540a1e1fa44b990c529363b994fce/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:09.713899 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06904896-8d94-4282-ad14-71e59fa5782e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:09.725699 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06904896-8d94-4282-ad14-71e59fa5782e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:09.782394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cb794-c32e-4699-9db5-b997fb3b0d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:09.783000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cb794-c32e-4699-9db5-b997fb3b0d65 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:09.783314 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cb794-c32e-4699-9db5-b997fb3b0d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:09.783590 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cb794-c32e-4699-9db5-b997fb3b0d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:09.784228 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043cb794-c32e-4699-9db5-b997fb3b0d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:09.785819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043cb794-c32e-4699-9db5-b997fb3b0d65 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:09.796518 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1364/2732] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:09.800806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc085c-35a2-4db5-b6da-dc3fdd3eda53 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:09.811753 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc085c-35a2-4db5-b6da-dc3fdd3eda53 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:09.816783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e23f5-1344-4787-a6e5-a13158bd6137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:09.817438 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e23f5-1344-4787-a6e5-a13158bd6137 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:09.817720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e23f5-1344-4787-a6e5-a13158bd6137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:09.817992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e23f5-1344-4787-a6e5-a13158bd6137 admin admin] PATH_INFO: `/v3/projects/8e718c511df34177bd5a8b58c8b0dd0c/users/893540a1e1fa44b990c529363b994fce/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:09.819923 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e23f5-1344-4787-a6e5-a13158bd6137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e718c511df34177bd5a8b58c8b0dd0c, user_id=893540a1e1fa44b990c529363b994fce, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:09.822562 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e23f5-1344-4787-a6e5-a13158bd6137 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:09.833120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1369/2733] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:03:09 2022] PUT /identity/v3/projects/8e718c511df34177bd5a8b58c8b0dd0c/users/893540a1e1fa44b990c529363b994fce/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:03:09.843487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e6241-d4f0-4865-b63e-c53c19edb285 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:09.843943 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e6241-d4f0-4865-b63e-c53c19edb285 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:09.844054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e6241-d4f0-4865-b63e-c53c19edb285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:09.872012 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-ea3e6241-d4f0-4865-b63e-c53c19edb285 None None] MFA Rules not processed for user `893540a1e1fa44b990c529363b994fce`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:09.873212 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ea3e6241-d4f0-4865-b63e-c53c19edb285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:09.940148 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1365/2734] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:09.952641 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e23f5-1344-4787-a6e5-a13158bd6137 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:09.954411 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2e0e23f5-1344-4787-a6e5-a13158bd6137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:10.031762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e23f5-1344-4787-a6e5-a13158bd6137 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:10.035496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19c0d4-1b5f-4e12-87a4-663a7debf59a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:10.036105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19c0d4-1b5f-4e12-87a4-663a7debf59a service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:10.036385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19c0d4-1b5f-4e12-87a4-663a7debf59a service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:10.036657 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19c0d4-1b5f-4e12-87a4-663a7debf59a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:10.046819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19c0d4-1b5f-4e12-87a4-663a7debf59a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:10.048273 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19c0d4-1b5f-4e12-87a4-663a7debf59a service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:10.058539 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1370/2735] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:03:11.029041 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e6241-d4f0-4865-b63e-c53c19edb285 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:11.038750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e6241-d4f0-4865-b63e-c53c19edb285 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:11.042328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2e2c9-ddcd-443f-8c77-fb81e989e3ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:11.042924 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2e2c9-ddcd-443f-8c77-fb81e989e3ca service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:11.043233 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2e2c9-ddcd-443f-8c77-fb81e989e3ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:11.043509 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2e2c9-ddcd-443f-8c77-fb81e989e3ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:11.099751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2e2c9-ddcd-443f-8c77-fb81e989e3ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:11.101181 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2e2c9-ddcd-443f-8c77-fb81e989e3ca service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:11.105852 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:11.110874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1366/2736] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:03:18.279964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19c0d4-1b5f-4e12-87a4-663a7debf59a service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:18.281359 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8b19c0d4-1b5f-4e12-87a4-663a7debf59a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:18.302330 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19c0d4-1b5f-4e12-87a4-663a7debf59a service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:18.306071 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7f01b-007f-49c1-94bd-dac0879870b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:18.306698 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7f01b-007f-49c1-94bd-dac0879870b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:18.306968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7f01b-007f-49c1-94bd-dac0879870b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:18.307274 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7f01b-007f-49c1-94bd-dac0879870b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:18.377936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7f01b-007f-49c1-94bd-dac0879870b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:18.379412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7f01b-007f-49c1-94bd-dac0879870b2 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:18.388997 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1371/2737] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:18 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:03:23.440356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2e2c9-ddcd-443f-8c77-fb81e989e3ca service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:23.522475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2e2c9-ddcd-443f-8c77-fb81e989e3ca service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:23.526895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35135cc7-9f27-4c93-9055-96859cfed877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:23.527559 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35135cc7-9f27-4c93-9055-96859cfed877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:23.527844 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35135cc7-9f27-4c93-9055-96859cfed877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:23.528121 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35135cc7-9f27-4c93-9055-96859cfed877 admin admin] PATH_INFO: `/v3/users/83065c4d26da46fe84de678cb277f3e6` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:23.529890 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35135cc7-9f27-4c93-9055-96859cfed877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83065c4d26da46fe84de678cb277f3e6)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:23.531855 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35135cc7-9f27-4c93-9055-96859cfed877 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:23.764134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-35135cc7-9f27-4c93-9055-96859cfed877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83065c4d26da46fe84de678cb277f3e6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:23.777714 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-35135cc7-9f27-4c93-9055-96859cfed877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83065c4d26da46fe84de678cb277f3e6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:23.786839 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-35135cc7-9f27-4c93-9055-96859cfed877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83065c4d26da46fe84de678cb277f3e6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:23.890757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-35135cc7-9f27-4c93-9055-96859cfed877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83065c4d26da46fe84de678cb277f3e6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:23.891412 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-35135cc7-9f27-4c93-9055-96859cfed877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83065c4d26da46fe84de678cb277f3e6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:23.898398 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1367/2738] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:03:23 2022] DELETE /identity/v3/users/83065c4d26da46fe84de678cb277f3e6 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:23.904763 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7caa195-bc6f-452d-a793-b5f06e15d411 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:23.905263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7caa195-bc6f-452d-a793-b5f06e15d411 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:23.905372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7caa195-bc6f-452d-a793-b5f06e15d411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:23.928237 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e7caa195-bc6f-452d-a793-b5f06e15d411 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:23.929572 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e7caa195-bc6f-452d-a793-b5f06e15d411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:23.952919 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35135cc7-9f27-4c93-9055-96859cfed877 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:23.954710 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-35135cc7-9f27-4c93-9055-96859cfed877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:23.993653 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:24.003772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1372/2739] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:24.013609 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7caa195-bc6f-452d-a793-b5f06e15d411 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:24.015030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e7caa195-bc6f-452d-a793-b5f06e15d411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:24.028177 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:24.058798 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35135cc7-9f27-4c93-9055-96859cfed877 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:24.064435 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91c6d6-5c39-4e71-863b-14086a4f3170 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:24.065082 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91c6d6-5c39-4e71-863b-14086a4f3170 service placement] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:24.065414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91c6d6-5c39-4e71-863b-14086a4f3170 service placement] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:24.065693 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91c6d6-5c39-4e71-863b-14086a4f3170 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:24.067571 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5d91c6d6-5c39-4e71-863b-14086a4f3170 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:24.076254 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7caa195-bc6f-452d-a793-b5f06e15d411 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:24.079527 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91c6d6-5c39-4e71-863b-14086a4f3170 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:24.081154 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91c6d6-5c39-4e71-863b-14086a4f3170 service placement] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:24.081256 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb0189-422d-4e86-be46-2b163fa3b886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:24.081868 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb0189-422d-4e86-be46-2b163fa3b886 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:24.082148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb0189-422d-4e86-be46-2b163fa3b886 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:24.082425 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb0189-422d-4e86-be46-2b163fa3b886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:24.092335 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb0189-422d-4e86-be46-2b163fa3b886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:24.092520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1368/2740] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:03:24.093952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb0189-422d-4e86-be46-2b163fa3b886 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:24.104056 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1373/2741] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:03:24.595187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91c6d6-5c39-4e71-863b-14086a4f3170 service placement] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:24.596640 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5d91c6d6-5c39-4e71-863b-14086a4f3170 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:24.610550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91c6d6-5c39-4e71-863b-14086a4f3170 service placement] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:24.615481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:24.616109 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:24.616407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:24.616670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a admin admin] PATH_INFO: `/v3/projects/9ec5468c0b96400a80d42cd122c95466` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:24.619011 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ec5468c0b96400a80d42cd122c95466)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:24.620962 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:24.766001 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec5468c0b96400a80d42cd122c95466', 'request_id': 'req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:24.774789 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec5468c0b96400a80d42cd122c95466', 'request_id': 'req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:24.839257 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec5468c0b96400a80d42cd122c95466', 'request_id': 'req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:24.851488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a admin admin] The token cache is being invalidate because project 9ec5468c0b96400a80d42cd122c95466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:03:24.851785 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:24.852449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:24.853937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1369/2742] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:03:24 2022] DELETE /identity/v3/projects/9ec5468c0b96400a80d42cd122c95466 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:24.858097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb0189-422d-4e86-be46-2b163fa3b886 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:24.868243 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb0189-422d-4e86-be46-2b163fa3b886 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:24.928027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6ee1d-0e25-4b05-b532-8a9e34b701df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:24.928701 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6ee1d-0e25-4b05-b532-8a9e34b701df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:24.928934 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6ee1d-0e25-4b05-b532-8a9e34b701df admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:24.929207 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6ee1d-0e25-4b05-b532-8a9e34b701df admin admin] PATH_INFO: `/v3/users/58c87c26d9904845bfed892b4706f62c` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:24.930381 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6ee1d-0e25-4b05-b532-8a9e34b701df admin admin] RBAC: Authorizing `identity:delete_user(user_id=58c87c26d9904845bfed892b4706f62c)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:24.932599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6ee1d-0e25-4b05-b532-8a9e34b701df admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:25.074867 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-42d6ee1d-0e25-4b05-b532-8a9e34b701df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58c87c26d9904845bfed892b4706f62c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:25.102018 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-42d6ee1d-0e25-4b05-b532-8a9e34b701df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58c87c26d9904845bfed892b4706f62c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:25.118877 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-42d6ee1d-0e25-4b05-b532-8a9e34b701df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58c87c26d9904845bfed892b4706f62c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:25.193607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-42d6ee1d-0e25-4b05-b532-8a9e34b701df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58c87c26d9904845bfed892b4706f62c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:25.194269 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-42d6ee1d-0e25-4b05-b532-8a9e34b701df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58c87c26d9904845bfed892b4706f62c'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:25.200807 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1374/2743] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:03:24 2022] DELETE /identity/v3/users/58c87c26d9904845bfed892b4706f62c => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:25.608858 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:25.611134 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:25.630887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43eee8-73d3-4691-81fd-98d0aca3ff7a admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:25.695795 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92722e-d66d-4871-8350-321167ab2894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:25.696779 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92722e-d66d-4871-8350-321167ab2894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:25.697250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92722e-d66d-4871-8350-321167ab2894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:25.697513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92722e-d66d-4871-8350-321167ab2894 admin admin] PATH_INFO: `/v3/projects/13003d22da7846e2acc01323cc30eca8` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:25.698653 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92722e-d66d-4871-8350-321167ab2894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13003d22da7846e2acc01323cc30eca8)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:25.700738 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92722e-d66d-4871-8350-321167ab2894 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:25.879407 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-bf92722e-d66d-4871-8350-321167ab2894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13003d22da7846e2acc01323cc30eca8', 'request_id': 'req-bf92722e-d66d-4871-8350-321167ab2894'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:25.883046 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:25.885141 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-bf92722e-d66d-4871-8350-321167ab2894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13003d22da7846e2acc01323cc30eca8', 'request_id': 'req-bf92722e-d66d-4871-8350-321167ab2894'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:25.975771 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-bf92722e-d66d-4871-8350-321167ab2894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13003d22da7846e2acc01323cc30eca8', 'request_id': 'req-bf92722e-d66d-4871-8350-321167ab2894'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:25.988035 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-bf92722e-d66d-4871-8350-321167ab2894 admin admin] The token cache is being invalidate because project 13003d22da7846e2acc01323cc30eca8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:03:25.989855 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-bf92722e-d66d-4871-8350-321167ab2894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:25.989855 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-bf92722e-d66d-4871-8350-321167ab2894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:25.991028 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1370/2744] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:03:25 2022] DELETE /identity/v3/projects/13003d22da7846e2acc01323cc30eca8 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:25.995106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6ee1d-0e25-4b05-b532-8a9e34b701df admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:25.998035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-42d6ee1d-0e25-4b05-b532-8a9e34b701df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:26.011162 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6ee1d-0e25-4b05-b532-8a9e34b701df admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:26.028196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:26.069195 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3861530-d090-4300-8a27-0acfd434549b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:26.069836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3861530-d090-4300-8a27-0acfd434549b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:26.070120 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3861530-d090-4300-8a27-0acfd434549b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:26.070394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3861530-d090-4300-8a27-0acfd434549b admin admin] PATH_INFO: `/v3/users/893540a1e1fa44b990c529363b994fce` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:26.071562 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3861530-d090-4300-8a27-0acfd434549b admin admin] RBAC: Authorizing `identity:delete_user(user_id=893540a1e1fa44b990c529363b994fce)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:26.073527 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3861530-d090-4300-8a27-0acfd434549b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:26.293621 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b3861530-d090-4300-8a27-0acfd434549b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '893540a1e1fa44b990c529363b994fce'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:26.305902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b3861530-d090-4300-8a27-0acfd434549b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '893540a1e1fa44b990c529363b994fce'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:26.313339 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b3861530-d090-4300-8a27-0acfd434549b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '893540a1e1fa44b990c529363b994fce'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:26.370678 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b3861530-d090-4300-8a27-0acfd434549b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '893540a1e1fa44b990c529363b994fce'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:26.371366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b3861530-d090-4300-8a27-0acfd434549b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '893540a1e1fa44b990c529363b994fce'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:26.378471 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1375/2745] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:03:25 2022] DELETE /identity/v3/users/893540a1e1fa44b990c529363b994fce => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:26.773469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92722e-d66d-4871-8350-321167ab2894 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:26.774999 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-bf92722e-d66d-4871-8350-321167ab2894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:26.792889 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92722e-d66d-4871-8350-321167ab2894 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:26.860172 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e071f-febf-4b05-bc72-052cb9257522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:26.860811 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e071f-febf-4b05-bc72-052cb9257522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:26.861091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e071f-febf-4b05-bc72-052cb9257522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:26.861363 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e071f-febf-4b05-bc72-052cb9257522 admin admin] PATH_INFO: `/v3/projects/8e718c511df34177bd5a8b58c8b0dd0c` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:26.862511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e071f-febf-4b05-bc72-052cb9257522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e718c511df34177bd5a8b58c8b0dd0c)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:26.864449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e071f-febf-4b05-bc72-052cb9257522 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:26.994007 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-656e071f-febf-4b05-bc72-052cb9257522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e718c511df34177bd5a8b58c8b0dd0c', 'request_id': 'req-656e071f-febf-4b05-bc72-052cb9257522'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:26.999597 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-656e071f-febf-4b05-bc72-052cb9257522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e718c511df34177bd5a8b58c8b0dd0c', 'request_id': 'req-656e071f-febf-4b05-bc72-052cb9257522'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:27.073779 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-656e071f-febf-4b05-bc72-052cb9257522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e718c511df34177bd5a8b58c8b0dd0c', 'request_id': 'req-656e071f-febf-4b05-bc72-052cb9257522'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:27.086518 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-656e071f-febf-4b05-bc72-052cb9257522 admin admin] The token cache is being invalidate because project 8e718c511df34177bd5a8b58c8b0dd0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:03:27.086811 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-656e071f-febf-4b05-bc72-052cb9257522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:27.087485 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-656e071f-febf-4b05-bc72-052cb9257522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:27.088985 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1371/2746] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:03:26 2022] DELETE /identity/v3/projects/8e718c511df34177bd5a8b58c8b0dd0c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:27.096546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d28804-5f8d-4214-9903-26f7246da6ff None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:27.097031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d28804-5f8d-4214-9903-26f7246da6ff None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:27.097123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d28804-5f8d-4214-9903-26f7246da6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:27.119420 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-99d28804-5f8d-4214-9903-26f7246da6ff None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:27.121673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-99d28804-5f8d-4214-9903-26f7246da6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:27.184916 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1376/2747] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:27.193416 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054b4c4-ba1a-4c41-b040-8ff9058cdfca None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:27.193703 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054b4c4-ba1a-4c41-b040-8ff9058cdfca None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:27.200799 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054b4c4-ba1a-4c41-b040-8ff9058cdfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:27.214422 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-7054b4c4-ba1a-4c41-b040-8ff9058cdfca None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:27.215702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7054b4c4-ba1a-4c41-b040-8ff9058cdfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:27.224464 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1372/2748] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:27.228717 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d28804-5f8d-4214-9903-26f7246da6ff None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:27.238870 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d28804-5f8d-4214-9903-26f7246da6ff None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:27.243732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b73a8-8ad4-4e2d-b726-f4f30fe56372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:27.244384 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b73a8-8ad4-4e2d-b726-f4f30fe56372 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:27.244614 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b73a8-8ad4-4e2d-b726-f4f30fe56372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:27.244887 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b73a8-8ad4-4e2d-b726-f4f30fe56372 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:27.245527 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b73a8-8ad4-4e2d-b726-f4f30fe56372 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:03:27.245831 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b73a8-8ad4-4e2d-b726-f4f30fe56372 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:27.247477 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b73a8-8ad4-4e2d-b726-f4f30fe56372 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:27.259133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1377/2749] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:03:27.264437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054b4c4-ba1a-4c41-b040-8ff9058cdfca None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:27.274184 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054b4c4-ba1a-4c41-b040-8ff9058cdfca None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:27.278939 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d3a4f-ec7a-482b-afe5-570a8336c6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:27.280121 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d3a4f-ec7a-482b-afe5-570a8336c6a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:27.280425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d3a4f-ec7a-482b-afe5-570a8336c6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:27.280725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d3a4f-ec7a-482b-afe5-570a8336c6a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:27.281362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d3a4f-ec7a-482b-afe5-570a8336c6a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:27.283297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d3a4f-ec7a-482b-afe5-570a8336c6a9 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:27.378988 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1373/2750] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:03:27 2022] POST /identity/v3/projects => generated 389 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:27.383721 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b73a8-8ad4-4e2d-b726-f4f30fe56372 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:27.394898 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b73a8-8ad4-4e2d-b726-f4f30fe56372 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:27.405802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d3a4f-ec7a-482b-afe5-570a8336c6a9 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:27.407616 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-280d3a4f-ec7a-482b-afe5-570a8336c6a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:27.421177 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:27.455588 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:27.455970 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88841b7e-fa5b-4428-97e6-34c8bfa811e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:27.456593 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88841b7e-fa5b-4428-97e6-34c8bfa811e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:27.456899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88841b7e-fa5b-4428-97e6-34c8bfa811e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:27.457175 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88841b7e-fa5b-4428-97e6-34c8bfa811e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:27.457854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88841b7e-fa5b-4428-97e6-34c8bfa811e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:27.459877 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88841b7e-fa5b-4428-97e6-34c8bfa811e7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:27.473295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d3a4f-ec7a-482b-afe5-570a8336c6a9 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:27.478039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070f37a-63d8-467e-ae49-3fb3e2cc117b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:27.478666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070f37a-63d8-467e-ae49-3fb3e2cc117b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:27.478947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070f37a-63d8-467e-ae49-3fb3e2cc117b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:27.479255 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070f37a-63d8-467e-ae49-3fb3e2cc117b admin admin] PATH_INFO: `/v3/users/585e2706642a48c9ac002d10e86de96e` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:27.484400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070f37a-63d8-467e-ae49-3fb3e2cc117b admin admin] RBAC: Authorizing `identity:delete_user(user_id=585e2706642a48c9ac002d10e86de96e)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:27.484400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070f37a-63d8-467e-ae49-3fb3e2cc117b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:27.525353 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1378/2751] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:03:27 2022] POST /identity/v3/users => generated 295 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:03:27.529853 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88841b7e-fa5b-4428-97e6-34c8bfa811e7 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:27.539746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88841b7e-fa5b-4428-97e6-34c8bfa811e7 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:27.544416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc8f8a-76a3-4790-977c-d7a7d2432768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:27.545666 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc8f8a-76a3-4790-977c-d7a7d2432768 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:27.545942 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc8f8a-76a3-4790-977c-d7a7d2432768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:27.546216 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc8f8a-76a3-4790-977c-d7a7d2432768 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:27.546838 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc8f8a-76a3-4790-977c-d7a7d2432768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:27.548444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc8f8a-76a3-4790-977c-d7a7d2432768 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:27.558209 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1379/2752] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:03:27.562233 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc8f8a-76a3-4790-977c-d7a7d2432768 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:27.572055 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc8f8a-76a3-4790-977c-d7a7d2432768 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:27.576654 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62658c69-8e43-41f1-8fe9-647b2515ae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:27.577235 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62658c69-8e43-41f1-8fe9-647b2515ae34 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:27.577511 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62658c69-8e43-41f1-8fe9-647b2515ae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:27.577784 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62658c69-8e43-41f1-8fe9-647b2515ae34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:27.578413 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62658c69-8e43-41f1-8fe9-647b2515ae34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:27.580040 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62658c69-8e43-41f1-8fe9-647b2515ae34 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:27.589936 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1380/2753] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:03:27.594384 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62658c69-8e43-41f1-8fe9-647b2515ae34 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:27.603947 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62658c69-8e43-41f1-8fe9-647b2515ae34 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:27.609093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87782905-cfb2-4700-90e0-e958c616cd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:27.609712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87782905-cfb2-4700-90e0-e958c616cd5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:27.616146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87782905-cfb2-4700-90e0-e958c616cd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:27.616146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87782905-cfb2-4700-90e0-e958c616cd5e admin admin] PATH_INFO: `/v3/projects/004857cf162b44be9d5388d591f8cfb3/users/4c0ce8f4a6174a6aa9b6a9855efdb0a3/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:27.624521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87782905-cfb2-4700-90e0-e958c616cd5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=004857cf162b44be9d5388d591f8cfb3, user_id=4c0ce8f4a6174a6aa9b6a9855efdb0a3, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:27.627140 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87782905-cfb2-4700-90e0-e958c616cd5e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:27.642098 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-0070f37a-63d8-467e-ae49-3fb3e2cc117b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '585e2706642a48c9ac002d10e86de96e'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:27.655389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-0070f37a-63d8-467e-ae49-3fb3e2cc117b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '585e2706642a48c9ac002d10e86de96e'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:27.662246 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-0070f37a-63d8-467e-ae49-3fb3e2cc117b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '585e2706642a48c9ac002d10e86de96e'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:27.691036 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1381/2754] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:03:27 2022] PUT /identity/v3/projects/004857cf162b44be9d5388d591f8cfb3/users/4c0ce8f4a6174a6aa9b6a9855efdb0a3/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:03:27.702417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b965e8a-fddb-4a21-a39f-e11201bcaf58 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:27.702709 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b965e8a-fddb-4a21-a39f-e11201bcaf58 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:27.702836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-0070f37a-63d8-467e-ae49-3fb3e2cc117b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '585e2706642a48c9ac002d10e86de96e'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:27.702978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b965e8a-fddb-4a21-a39f-e11201bcaf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:27.703506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-0070f37a-63d8-467e-ae49-3fb3e2cc117b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '585e2706642a48c9ac002d10e86de96e'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:27.710102 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1374/2755] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:03:27 2022] DELETE /identity/v3/users/585e2706642a48c9ac002d10e86de96e => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:03:27.716448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dfec7-700a-4ed5-a075-d8f255f2d1f1 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:27.717247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dfec7-700a-4ed5-a075-d8f255f2d1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:27.717631 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dfec7-700a-4ed5-a075-d8f255f2d1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:27.736396 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-5b965e8a-fddb-4a21-a39f-e11201bcaf58 None None] MFA Rules not processed for user `4c0ce8f4a6174a6aa9b6a9855efdb0a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:27.737670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5b965e8a-fddb-4a21-a39f-e11201bcaf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:27.739552 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-af3dfec7-700a-4ed5-a075-d8f255f2d1f1 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:27.740856 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-af3dfec7-700a-4ed5-a075-d8f255f2d1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:27.805063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1375/2756] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:03:27.808998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1382/2757] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:03:27.809569 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dfec7-700a-4ed5-a075-d8f255f2d1f1 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:27.811337 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-af3dfec7-700a-4ed5-a075-d8f255f2d1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:27.815546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b965e8a-fddb-4a21-a39f-e11201bcaf58 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:27.817107 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5b965e8a-fddb-4a21-a39f-e11201bcaf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:27.822966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dfec7-700a-4ed5-a075-d8f255f2d1f1 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:27.828217 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5fe40-fb2e-4d08-8ed1-19488aab2520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:27.828832 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5fe40-fb2e-4d08-8ed1-19488aab2520 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:27.829112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5fe40-fb2e-4d08-8ed1-19488aab2520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:27.829389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5fe40-fb2e-4d08-8ed1-19488aab2520 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:27.830047 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5fe40-fb2e-4d08-8ed1-19488aab2520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:27.831991 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5fe40-fb2e-4d08-8ed1-19488aab2520 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:27.876713 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b965e8a-fddb-4a21-a39f-e11201bcaf58 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:27.880279 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974e031-3d2e-43db-9369-1db332fd3c5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:27.880880 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974e031-3d2e-43db-9369-1db332fd3c5e service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:27.881158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974e031-3d2e-43db-9369-1db332fd3c5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:27.881425 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974e031-3d2e-43db-9369-1db332fd3c5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:27.890433 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974e031-3d2e-43db-9369-1db332fd3c5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:27.893921 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974e031-3d2e-43db-9369-1db332fd3c5e service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:27.901439 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1383/2758] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:03:27.949774 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1376/2759] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:03:27 2022] POST /identity/v3/projects => generated 389 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:27.954393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974e031-3d2e-43db-9369-1db332fd3c5e service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:27.959365 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:27.969213 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974e031-3d2e-43db-9369-1db332fd3c5e service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:28.034672 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e3247-cd54-4198-97a3-35c7fc333c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:28.035316 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e3247-cd54-4198-97a3-35c7fc333c5f admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:28.035604 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e3247-cd54-4198-97a3-35c7fc333c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:28.036012 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e3247-cd54-4198-97a3-35c7fc333c5f admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:28.036660 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e3247-cd54-4198-97a3-35c7fc333c5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:28.038566 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e3247-cd54-4198-97a3-35c7fc333c5f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:28.160628 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1384/2760] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:03:27 2022] POST /identity/v3/users => generated 301 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:03:28.164592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5fe40-fb2e-4d08-8ed1-19488aab2520 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:28.175343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5fe40-fb2e-4d08-8ed1-19488aab2520 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:28.180095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706d89d-9233-4778-8224-ba65e162e2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:28.180704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706d89d-9233-4778-8224-ba65e162e2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:28.180979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706d89d-9233-4778-8224-ba65e162e2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:28.181251 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706d89d-9233-4778-8224-ba65e162e2b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:28.181894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706d89d-9233-4778-8224-ba65e162e2b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:28.183513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706d89d-9233-4778-8224-ba65e162e2b9 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:28.194545 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1377/2761] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:28.198786 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e3247-cd54-4198-97a3-35c7fc333c5f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:28.209708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e3247-cd54-4198-97a3-35c7fc333c5f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:28.214469 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d325ed-8b01-4d28-895c-9711ef0e8ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:28.217480 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d325ed-8b01-4d28-895c-9711ef0e8ac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:28.217480 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d325ed-8b01-4d28-895c-9711ef0e8ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:28.217480 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d325ed-8b01-4d28-895c-9711ef0e8ac9 admin admin] PATH_INFO: `/v3/projects/c3e119d02e6d41f091d591dfa50524e4/users/6e0fc0b7a8fd4218b5c4b87af5c0e74d/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:28.232035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d325ed-8b01-4d28-895c-9711ef0e8ac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e119d02e6d41f091d591dfa50524e4, user_id=6e0fc0b7a8fd4218b5c4b87af5c0e74d, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:28.239970 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d325ed-8b01-4d28-895c-9711ef0e8ac9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:28.302360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1385/2762] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:03:28 2022] PUT /identity/v3/projects/c3e119d02e6d41f091d591dfa50524e4/users/6e0fc0b7a8fd4218b5c4b87af5c0e74d/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:28.306446 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706d89d-9233-4778-8224-ba65e162e2b9 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:28.318865 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706d89d-9233-4778-8224-ba65e162e2b9 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:28.328066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d325ed-8b01-4d28-895c-9711ef0e8ac9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:28.329893 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-45d325ed-8b01-4d28-895c-9711ef0e8ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:28.355551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:28.384525 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63819842-6b3c-493a-a244-94a9d7a3ee7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:28.385143 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63819842-6b3c-493a-a244-94a9d7a3ee7b admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:28.385425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63819842-6b3c-493a-a244-94a9d7a3ee7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:28.385700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63819842-6b3c-493a-a244-94a9d7a3ee7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:28.386766 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63819842-6b3c-493a-a244-94a9d7a3ee7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:28.388156 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63819842-6b3c-493a-a244-94a9d7a3ee7b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:28.399153 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1378/2763] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:28.403289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63819842-6b3c-493a-a244-94a9d7a3ee7b admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:28.413462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63819842-6b3c-493a-a244-94a9d7a3ee7b admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:28.413684 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d325ed-8b01-4d28-895c-9711ef0e8ac9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:28.418838 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68114dd4-c966-4e49-8fc2-95890ff85751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:28.419052 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:28.419530 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68114dd4-c966-4e49-8fc2-95890ff85751 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:28.419787 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:28.419951 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68114dd4-c966-4e49-8fc2-95890ff85751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:28.425546 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:28.425656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68114dd4-c966-4e49-8fc2-95890ff85751 admin admin] PATH_INFO: `/v3/projects/c3e119d02e6d41f091d591dfa50524e4/users/6e0fc0b7a8fd4218b5c4b87af5c0e74d/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:28.425779 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a admin admin] PATH_INFO: `/v3/projects/6fa4dc2f16ff4ae8903f936984f9ba53` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:28.425779 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fa4dc2f16ff4ae8903f936984f9ba53)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:28.425937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68114dd4-c966-4e49-8fc2-95890ff85751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e119d02e6d41f091d591dfa50524e4, user_id=6e0fc0b7a8fd4218b5c4b87af5c0e74d, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:28.426035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:28.426118 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68114dd4-c966-4e49-8fc2-95890ff85751 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:28.437506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1379/2764] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:03:28 2022] PUT /identity/v3/projects/c3e119d02e6d41f091d591dfa50524e4/users/6e0fc0b7a8fd4218b5c4b87af5c0e74d/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:03:28.449655 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fad3ad-b063-4c9d-b4b4-c643d58471cb None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:28.449936 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fad3ad-b063-4c9d-b4b4-c643d58471cb None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:28.450204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fad3ad-b063-4c9d-b4b4-c643d58471cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:28.477657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-51fad3ad-b063-4c9d-b4b4-c643d58471cb None None] MFA Rules not processed for user `6e0fc0b7a8fd4218b5c4b87af5c0e74d`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:28.478859 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-51fad3ad-b063-4c9d-b4b4-c643d58471cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:28.545934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1380/2765] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:28 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:28.556823 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fad3ad-b063-4c9d-b4b4-c643d58471cb None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:28.558163 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fa4dc2f16ff4ae8903f936984f9ba53', 'request_id': 'req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:28.558323 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-51fad3ad-b063-4c9d-b4b4-c643d58471cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:28.617114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fad3ad-b063-4c9d-b4b4-c643d58471cb None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:28.627932 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fa4dc2f16ff4ae8903f936984f9ba53', 'request_id': 'req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:28.635383 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fa4dc2f16ff4ae8903f936984f9ba53', 'request_id': 'req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:28.643718 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:28.648988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a admin admin] The token cache is being invalidate because project 6fa4dc2f16ff4ae8903f936984f9ba53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:03:28.649383 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:28.650372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:28.652374 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1386/2766] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:03:28 2022] DELETE /identity/v3/projects/6fa4dc2f16ff4ae8903f936984f9ba53 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:03:28.671278 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1de3c-70e3-4a4a-bfd3-266ec1d88fd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:28.671915 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1de3c-70e3-4a4a-bfd3-266ec1d88fd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:28.672196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1de3c-70e3-4a4a-bfd3-266ec1d88fd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:28.672479 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1de3c-70e3-4a4a-bfd3-266ec1d88fd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:28.674449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-31e1de3c-70e3-4a4a-bfd3-266ec1d88fd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:28.685388 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:28.746670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e1de3c-70e3-4a4a-bfd3-266ec1d88fd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:28.749031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e1de3c-70e3-4a4a-bfd3-266ec1d88fd5 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:28.760396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1381/2767] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:28 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:03:28.915822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:28.917361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:28.981720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5585d9c-2b9a-40dd-8e32-9a75a3af197a admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:28.986955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3e087-5423-4202-aa35-7486f1f3247e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:28.987607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3e087-5423-4202-aa35-7486f1f3247e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:28.987889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3e087-5423-4202-aa35-7486f1f3247e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:28.988161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3e087-5423-4202-aa35-7486f1f3247e admin admin] PATH_INFO: `/v3/users/4c0ce8f4a6174a6aa9b6a9855efdb0a3` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:28.989273 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3e087-5423-4202-aa35-7486f1f3247e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c0ce8f4a6174a6aa9b6a9855efdb0a3)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:28.991228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3e087-5423-4202-aa35-7486f1f3247e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:29.248139 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ade3e087-5423-4202-aa35-7486f1f3247e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c0ce8f4a6174a6aa9b6a9855efdb0a3'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:29.260899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ade3e087-5423-4202-aa35-7486f1f3247e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c0ce8f4a6174a6aa9b6a9855efdb0a3'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:29.268009 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ade3e087-5423-4202-aa35-7486f1f3247e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c0ce8f4a6174a6aa9b6a9855efdb0a3'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:29.370728 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ade3e087-5423-4202-aa35-7486f1f3247e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c0ce8f4a6174a6aa9b6a9855efdb0a3'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:29.371505 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-ade3e087-5423-4202-aa35-7486f1f3247e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c0ce8f4a6174a6aa9b6a9855efdb0a3'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:29.378419 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1387/2768] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:03:28 2022] DELETE /identity/v3/users/4c0ce8f4a6174a6aa9b6a9855efdb0a3 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:29.382920 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a555c-5b31-4bb7-bc11-8cbadb78b7cd None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:29.383410 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a555c-5b31-4bb7-bc11-8cbadb78b7cd None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:29.383519 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a555c-5b31-4bb7-bc11-8cbadb78b7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:29.406630 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-a26a555c-5b31-4bb7-bc11-8cbadb78b7cd None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:29.408070 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a26a555c-5b31-4bb7-bc11-8cbadb78b7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:29.469295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1382/2769] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:29.479250 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3e087-5423-4202-aa35-7486f1f3247e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:29.483873 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ade3e087-5423-4202-aa35-7486f1f3247e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:29.537634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3e087-5423-4202-aa35-7486f1f3247e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:29.541139 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94547808-26c6-436a-9ab5-b3ff5bd875cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:29.541745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94547808-26c6-436a-9ab5-b3ff5bd875cd service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:29.542027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94547808-26c6-436a-9ab5-b3ff5bd875cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:29.542306 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94547808-26c6-436a-9ab5-b3ff5bd875cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:29.551692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94547808-26c6-436a-9ab5-b3ff5bd875cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:29.553158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94547808-26c6-436a-9ab5-b3ff5bd875cd service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:29.562756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1388/2770] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:03:29.972113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a555c-5b31-4bb7-bc11-8cbadb78b7cd None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:29.973753 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a26a555c-5b31-4bb7-bc11-8cbadb78b7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:29.987663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a555c-5b31-4bb7-bc11-8cbadb78b7cd None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:29.993157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1925c9-e8bc-418c-8d76-a7654e1f422a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:29.994093 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1925c9-e8bc-418c-8d76-a7654e1f422a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:29.994376 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1925c9-e8bc-418c-8d76-a7654e1f422a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:29.994649 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1925c9-e8bc-418c-8d76-a7654e1f422a admin admin] PATH_INFO: `/v3/projects/004857cf162b44be9d5388d591f8cfb3` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:29.995877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1925c9-e8bc-418c-8d76-a7654e1f422a admin admin] RBAC: Authorizing `identity:delete_project(project_id=004857cf162b44be9d5388d591f8cfb3)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:29.998469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1925c9-e8bc-418c-8d76-a7654e1f422a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:30.108048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-9e1925c9-e8bc-418c-8d76-a7654e1f422a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '004857cf162b44be9d5388d591f8cfb3', 'request_id': 'req-9e1925c9-e8bc-418c-8d76-a7654e1f422a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:30.113975 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-9e1925c9-e8bc-418c-8d76-a7654e1f422a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '004857cf162b44be9d5388d591f8cfb3', 'request_id': 'req-9e1925c9-e8bc-418c-8d76-a7654e1f422a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:30.205095 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-9e1925c9-e8bc-418c-8d76-a7654e1f422a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '004857cf162b44be9d5388d591f8cfb3', 'request_id': 'req-9e1925c9-e8bc-418c-8d76-a7654e1f422a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:30.216936 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-9e1925c9-e8bc-418c-8d76-a7654e1f422a admin admin] The token cache is being invalidate because project 004857cf162b44be9d5388d591f8cfb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:03:30.217221 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-9e1925c9-e8bc-418c-8d76-a7654e1f422a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:30.217837 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-9e1925c9-e8bc-418c-8d76-a7654e1f422a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:30.219362 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1383/2771] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:03:29 2022] DELETE /identity/v3/projects/004857cf162b44be9d5388d591f8cfb3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:30.223456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94547808-26c6-436a-9ab5-b3ff5bd875cd service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:30.233535 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94547808-26c6-436a-9ab5-b3ff5bd875cd service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:30.250264 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:30.289079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc79e2-3ecc-4843-a843-5b4687738098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:30.289705 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc79e2-3ecc-4843-a843-5b4687738098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:30.289984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc79e2-3ecc-4843-a843-5b4687738098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:30.290255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc79e2-3ecc-4843-a843-5b4687738098 admin admin] PATH_INFO: `/v3/users/6e0fc0b7a8fd4218b5c4b87af5c0e74d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:30.291391 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc79e2-3ecc-4843-a843-5b4687738098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e0fc0b7a8fd4218b5c4b87af5c0e74d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:30.293287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc79e2-3ecc-4843-a843-5b4687738098 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:30.442429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-37bc79e2-3ecc-4843-a843-5b4687738098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e0fc0b7a8fd4218b5c4b87af5c0e74d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:30.454572 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-37bc79e2-3ecc-4843-a843-5b4687738098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e0fc0b7a8fd4218b5c4b87af5c0e74d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:30.461521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-37bc79e2-3ecc-4843-a843-5b4687738098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e0fc0b7a8fd4218b5c4b87af5c0e74d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:30.530480 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-37bc79e2-3ecc-4843-a843-5b4687738098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e0fc0b7a8fd4218b5c4b87af5c0e74d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:30.531097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-37bc79e2-3ecc-4843-a843-5b4687738098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e0fc0b7a8fd4218b5c4b87af5c0e74d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:30.537312 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1389/2772] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:03:30 2022] DELETE /identity/v3/users/6e0fc0b7a8fd4218b5c4b87af5c0e74d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:30.943409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1925c9-e8bc-418c-8d76-a7654e1f422a admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:30.945080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-9e1925c9-e8bc-418c-8d76-a7654e1f422a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:30.959828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1925c9-e8bc-418c-8d76-a7654e1f422a admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:30.995372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:31.028463 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b239041-c01b-4f29-885a-e5123b3779a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:31.029700 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b239041-c01b-4f29-885a-e5123b3779a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:31.030011 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b239041-c01b-4f29-885a-e5123b3779a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:31.030296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b239041-c01b-4f29-885a-e5123b3779a6 admin admin] PATH_INFO: `/v3/projects/c3e119d02e6d41f091d591dfa50524e4` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:31.032393 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b239041-c01b-4f29-885a-e5123b3779a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3e119d02e6d41f091d591dfa50524e4)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:31.034311 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b239041-c01b-4f29-885a-e5123b3779a6 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:31.155014 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-7b239041-c01b-4f29-885a-e5123b3779a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3e119d02e6d41f091d591dfa50524e4', 'request_id': 'req-7b239041-c01b-4f29-885a-e5123b3779a6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:31.160637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-7b239041-c01b-4f29-885a-e5123b3779a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3e119d02e6d41f091d591dfa50524e4', 'request_id': 'req-7b239041-c01b-4f29-885a-e5123b3779a6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:31.233502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-7b239041-c01b-4f29-885a-e5123b3779a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3e119d02e6d41f091d591dfa50524e4', 'request_id': 'req-7b239041-c01b-4f29-885a-e5123b3779a6'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:31.245322 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-7b239041-c01b-4f29-885a-e5123b3779a6 admin admin] The token cache is being invalidate because project c3e119d02e6d41f091d591dfa50524e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:03:31.245609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-7b239041-c01b-4f29-885a-e5123b3779a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:31.246237 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-7b239041-c01b-4f29-885a-e5123b3779a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:31.247750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1384/2773] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:03:30 2022] DELETE /identity/v3/projects/c3e119d02e6d41f091d591dfa50524e4 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:31.252748 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634453b-a468-4e48-b7fd-ed31746b913a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:31.253254 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634453b-a468-4e48-b7fd-ed31746b913a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:31.253343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634453b-a468-4e48-b7fd-ed31746b913a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:31.273577 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-7634453b-a468-4e48-b7fd-ed31746b913a None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:31.274777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-7634453b-a468-4e48-b7fd-ed31746b913a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:31.335549 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1390/2774] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:31.343629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9418c3-4d95-4195-833b-a30da30b6617 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:31.349453 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9418c3-4d95-4195-833b-a30da30b6617 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:31.349453 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9418c3-4d95-4195-833b-a30da30b6617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:31.364684 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-ec9418c3-4d95-4195-833b-a30da30b6617 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:31.368409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ec9418c3-4d95-4195-833b-a30da30b6617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:31.382218 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1385/2775] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:31.398835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634453b-a468-4e48-b7fd-ed31746b913a None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:31.408684 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634453b-a468-4e48-b7fd-ed31746b913a None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:31.413624 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29e920-3db1-471f-b7e4-908596339e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:31.414244 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29e920-3db1-471f-b7e4-908596339e9f admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:31.414527 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29e920-3db1-471f-b7e4-908596339e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:31.414963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29e920-3db1-471f-b7e4-908596339e9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:31.415551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29e920-3db1-471f-b7e4-908596339e9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:03:31.415853 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29e920-3db1-471f-b7e4-908596339e9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:31.417464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29e920-3db1-471f-b7e4-908596339e9f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:31.430436 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1391/2776] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:03:31.434836 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9418c3-4d95-4195-833b-a30da30b6617 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:31.444823 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9418c3-4d95-4195-833b-a30da30b6617 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:31.449903 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4848e4-27f0-418d-93b5-f665fa35448d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:31.450515 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4848e4-27f0-418d-93b5-f665fa35448d admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:31.450791 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4848e4-27f0-418d-93b5-f665fa35448d admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:31.451708 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4848e4-27f0-418d-93b5-f665fa35448d admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:31.456934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4848e4-27f0-418d-93b5-f665fa35448d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:31.456934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4848e4-27f0-418d-93b5-f665fa35448d admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:31.554983 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1386/2777] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:03:31 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:03:31.559427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29e920-3db1-471f-b7e4-908596339e9f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:31.570592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29e920-3db1-471f-b7e4-908596339e9f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:31.594639 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:31.626493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a15c1-bcd0-4648-be69-59dca41f6ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:31.627109 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a15c1-bcd0-4648-be69-59dca41f6ccc admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:31.627427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a15c1-bcd0-4648-be69-59dca41f6ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:31.627706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a15c1-bcd0-4648-be69-59dca41f6ccc admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:31.628338 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a15c1-bcd0-4648-be69-59dca41f6ccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:31.634556 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a15c1-bcd0-4648-be69-59dca41f6ccc admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:31.724956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1392/2778] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:03:31 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:31.729041 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4848e4-27f0-418d-93b5-f665fa35448d admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:31.740201 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4848e4-27f0-418d-93b5-f665fa35448d admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:31.744922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b150fe-b4b2-42ec-bdcd-275209fbe75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:31.745522 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b150fe-b4b2-42ec-bdcd-275209fbe75d admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:31.745865 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b150fe-b4b2-42ec-bdcd-275209fbe75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:31.746143 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b150fe-b4b2-42ec-bdcd-275209fbe75d admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:31.746788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b150fe-b4b2-42ec-bdcd-275209fbe75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:31.749099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b150fe-b4b2-42ec-bdcd-275209fbe75d admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:31.764001 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1387/2779] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:31.768952 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a15c1-bcd0-4648-be69-59dca41f6ccc admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:31.779017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a15c1-bcd0-4648-be69-59dca41f6ccc admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:31.784032 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97910ec8-2828-41ef-9277-005560f58e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:31.784646 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97910ec8-2828-41ef-9277-005560f58e87 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:31.784929 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97910ec8-2828-41ef-9277-005560f58e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:31.785205 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97910ec8-2828-41ef-9277-005560f58e87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:31.785987 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97910ec8-2828-41ef-9277-005560f58e87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:31.788481 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97910ec8-2828-41ef-9277-005560f58e87 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:31.799102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1393/2780] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:31.803405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b150fe-b4b2-42ec-bdcd-275209fbe75d admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:31.814044 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b150fe-b4b2-42ec-bdcd-275209fbe75d admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:31.818638 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3c27f-bbe5-4715-a3c9-86801a3ab7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:31.824268 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3c27f-bbe5-4715-a3c9-86801a3ab7a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:31.824268 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3c27f-bbe5-4715-a3c9-86801a3ab7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:31.824268 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3c27f-bbe5-4715-a3c9-86801a3ab7a7 admin admin] PATH_INFO: `/v3/projects/d37bc6402ef64c3fb1630e8122487a6f/users/2450c53f5e694a21b0fdaf5d8fc15834/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:31.834258 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3c27f-bbe5-4715-a3c9-86801a3ab7a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d37bc6402ef64c3fb1630e8122487a6f, user_id=2450c53f5e694a21b0fdaf5d8fc15834, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:31.837035 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3c27f-bbe5-4715-a3c9-86801a3ab7a7 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:31.896920 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1388/2781] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:03:31 2022] PUT /identity/v3/projects/d37bc6402ef64c3fb1630e8122487a6f/users/2450c53f5e694a21b0fdaf5d8fc15834/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:31.908381 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09439f39-d948-4f11-8e29-579ffbbee56f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:31.908672 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09439f39-d948-4f11-8e29-579ffbbee56f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:31.908954 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09439f39-d948-4f11-8e29-579ffbbee56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:31.936745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-09439f39-d948-4f11-8e29-579ffbbee56f None None] MFA Rules not processed for user `2450c53f5e694a21b0fdaf5d8fc15834`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:31.937962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-09439f39-d948-4f11-8e29-579ffbbee56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:32.002048 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1394/2782] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:32.006744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3c27f-bbe5-4715-a3c9-86801a3ab7a7 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:32.067606 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:32.072686 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3c27f-bbe5-4715-a3c9-86801a3ab7a7 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:32.078681 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1824fe1-0732-43d3-8390-0c1757746765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:32.079218 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1824fe1-0732-43d3-8390-0c1757746765 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:32.079475 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1824fe1-0732-43d3-8390-0c1757746765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:32.079749 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1824fe1-0732-43d3-8390-0c1757746765 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:32.080392 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1824fe1-0732-43d3-8390-0c1757746765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:32.082445 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1824fe1-0732-43d3-8390-0c1757746765 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:32.189785 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1389/2783] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:03:32 2022] POST /identity/v3/projects => generated 379 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:32.194129 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09439f39-d948-4f11-8e29-579ffbbee56f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:32.205636 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09439f39-d948-4f11-8e29-579ffbbee56f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:32.262637 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a7335-9c61-46be-8867-072fc4568d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:32.263355 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a7335-9c61-46be-8867-072fc4568d0d admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:32.264436 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a7335-9c61-46be-8867-072fc4568d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:32.264436 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a7335-9c61-46be-8867-072fc4568d0d admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:32.265022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a7335-9c61-46be-8867-072fc4568d0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:32.266949 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a7335-9c61-46be-8867-072fc4568d0d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:32.354366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1395/2784] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:03:32 2022] POST /identity/v3/users => generated 290 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:32.358433 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1824fe1-0732-43d3-8390-0c1757746765 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:32.369227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1824fe1-0732-43d3-8390-0c1757746765 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:32.374480 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570591b0-40e2-4eaa-af36-29d2b896a3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:32.375085 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570591b0-40e2-4eaa-af36-29d2b896a3bd admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:32.375398 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570591b0-40e2-4eaa-af36-29d2b896a3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:32.375675 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570591b0-40e2-4eaa-af36-29d2b896a3bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:32.376320 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570591b0-40e2-4eaa-af36-29d2b896a3bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:32.377933 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570591b0-40e2-4eaa-af36-29d2b896a3bd admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:32.389177 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1390/2785] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:32.393774 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a7335-9c61-46be-8867-072fc4568d0d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:32.404261 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a7335-9c61-46be-8867-072fc4568d0d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:32.409427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c3acf-103e-45a1-a7e7-cb7455bdcc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:32.410043 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c3acf-103e-45a1-a7e7-cb7455bdcc06 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:32.410351 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c3acf-103e-45a1-a7e7-cb7455bdcc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:32.410659 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c3acf-103e-45a1-a7e7-cb7455bdcc06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:32.411351 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c3acf-103e-45a1-a7e7-cb7455bdcc06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:32.412994 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c3acf-103e-45a1-a7e7-cb7455bdcc06 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:32.424221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1396/2786] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:32.428927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570591b0-40e2-4eaa-af36-29d2b896a3bd admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:32.440129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570591b0-40e2-4eaa-af36-29d2b896a3bd admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:32.444849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2df793-bedb-46ec-bba1-893486260bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:32.445504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2df793-bedb-46ec-bba1-893486260bb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:32.445785 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2df793-bedb-46ec-bba1-893486260bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:32.446063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2df793-bedb-46ec-bba1-893486260bb5 admin admin] PATH_INFO: `/v3/projects/033d41adb04748a49de907282c56b877/users/424554b896ac42ddb7625682405da5c9/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:32.460627 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2df793-bedb-46ec-bba1-893486260bb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=033d41adb04748a49de907282c56b877, user_id=424554b896ac42ddb7625682405da5c9, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:32.463323 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2df793-bedb-46ec-bba1-893486260bb5 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:32.541631 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1391/2787] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:03:32 2022] PUT /identity/v3/projects/033d41adb04748a49de907282c56b877/users/424554b896ac42ddb7625682405da5c9/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:32.553609 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc8cec-61a3-4832-b89b-a0676b07c774 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:32.553902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc8cec-61a3-4832-b89b-a0676b07c774 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:32.554181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc8cec-61a3-4832-b89b-a0676b07c774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:32.580591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:32.583450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-fffc8cec-61a3-4832-b89b-a0676b07c774 None None] MFA Rules not processed for user `424554b896ac42ddb7625682405da5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:32.584918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-fffc8cec-61a3-4832-b89b-a0676b07c774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:32.650835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1397/2788] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:32.656192 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2df793-bedb-46ec-bba1-893486260bb5 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:32.717189 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2df793-bedb-46ec-bba1-893486260bb5 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:32.722351 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb84b8-0a9e-4b18-a32f-19ea2219b91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:32.722956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb84b8-0a9e-4b18-a32f-19ea2219b91e admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:32.723272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb84b8-0a9e-4b18-a32f-19ea2219b91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:32.723551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb84b8-0a9e-4b18-a32f-19ea2219b91e admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:32.724172 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb84b8-0a9e-4b18-a32f-19ea2219b91e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:32.726077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb84b8-0a9e-4b18-a32f-19ea2219b91e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:32.805642 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1392/2789] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:03:32 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:32.809938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc8cec-61a3-4832-b89b-a0676b07c774 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:32.821499 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc8cec-61a3-4832-b89b-a0676b07c774 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:32.877706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50187d-571b-471b-b6fb-e53dfa5c8b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:32.878321 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50187d-571b-471b-b6fb-e53dfa5c8b2f admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:32.878604 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50187d-571b-471b-b6fb-e53dfa5c8b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:32.878880 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50187d-571b-471b-b6fb-e53dfa5c8b2f admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:32.879557 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50187d-571b-471b-b6fb-e53dfa5c8b2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:32.881458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50187d-571b-471b-b6fb-e53dfa5c8b2f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:32.982988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1398/2790] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:03:32 2022] POST /identity/v3/users => generated 296 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:32.986800 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb84b8-0a9e-4b18-a32f-19ea2219b91e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:32.997330 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb84b8-0a9e-4b18-a32f-19ea2219b91e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:33.002741 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488ea0b-184f-4d67-ac7e-20bc652aa287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:33.003436 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488ea0b-184f-4d67-ac7e-20bc652aa287 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:33.003716 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488ea0b-184f-4d67-ac7e-20bc652aa287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:33.003987 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488ea0b-184f-4d67-ac7e-20bc652aa287 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:33.004627 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488ea0b-184f-4d67-ac7e-20bc652aa287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:33.006214 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488ea0b-184f-4d67-ac7e-20bc652aa287 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:33.017570 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1393/2791] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:33.021752 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50187d-571b-471b-b6fb-e53dfa5c8b2f admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:33.032096 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50187d-571b-471b-b6fb-e53dfa5c8b2f admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:33.036812 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e665b9d-1d28-4d16-a4f9-00f9cecd9d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:33.037460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e665b9d-1d28-4d16-a4f9-00f9cecd9d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:33.037765 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e665b9d-1d28-4d16-a4f9-00f9cecd9d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:33.038038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e665b9d-1d28-4d16-a4f9-00f9cecd9d48 admin admin] PATH_INFO: `/v3/projects/3415db307a2c43cab0fab4ab3fc71347/users/2252eb6e54a044c5ad27b02bfbccc1f7/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:33.052667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e665b9d-1d28-4d16-a4f9-00f9cecd9d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3415db307a2c43cab0fab4ab3fc71347, user_id=2252eb6e54a044c5ad27b02bfbccc1f7, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:33.055365 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e665b9d-1d28-4d16-a4f9-00f9cecd9d48 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:33.131443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1399/2792] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:03:33 2022] PUT /identity/v3/projects/3415db307a2c43cab0fab4ab3fc71347/users/2252eb6e54a044c5ad27b02bfbccc1f7/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:33.135412 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488ea0b-184f-4d67-ac7e-20bc652aa287 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:33.151276 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488ea0b-184f-4d67-ac7e-20bc652aa287 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:33.202985 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfca88-6bbf-44a0-85eb-6730fbb32098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:33.203665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfca88-6bbf-44a0-85eb-6730fbb32098 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:33.203950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfca88-6bbf-44a0-85eb-6730fbb32098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:33.204223 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfca88-6bbf-44a0-85eb-6730fbb32098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:33.204862 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cfca88-6bbf-44a0-85eb-6730fbb32098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:33.206455 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cfca88-6bbf-44a0-85eb-6730fbb32098 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:33.216760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1394/2793] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:33.221231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e665b9d-1d28-4d16-a4f9-00f9cecd9d48 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:33.231656 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e665b9d-1d28-4d16-a4f9-00f9cecd9d48 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:33.236837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bf225-a242-46a6-b5c3-3b96e8c92f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:33.237433 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bf225-a242-46a6-b5c3-3b96e8c92f16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:33.237710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bf225-a242-46a6-b5c3-3b96e8c92f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:33.237982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bf225-a242-46a6-b5c3-3b96e8c92f16 admin admin] PATH_INFO: `/v3/projects/3415db307a2c43cab0fab4ab3fc71347/users/2252eb6e54a044c5ad27b02bfbccc1f7/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:33.239965 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bf225-a242-46a6-b5c3-3b96e8c92f16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3415db307a2c43cab0fab4ab3fc71347, user_id=2252eb6e54a044c5ad27b02bfbccc1f7, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:33.242628 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bf225-a242-46a6-b5c3-3b96e8c92f16 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:33.253643 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1400/2794] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:03:33 2022] PUT /identity/v3/projects/3415db307a2c43cab0fab4ab3fc71347/users/2252eb6e54a044c5ad27b02bfbccc1f7/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:33.263871 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf784aa-add0-4637-b5c1-807a82d9eba4 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:33.264348 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf784aa-add0-4637-b5c1-807a82d9eba4 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:33.264814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf784aa-add0-4637-b5c1-807a82d9eba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:33.294194 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-6bf784aa-add0-4637-b5c1-807a82d9eba4 None None] MFA Rules not processed for user `2252eb6e54a044c5ad27b02bfbccc1f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:33.295961 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-6bf784aa-add0-4637-b5c1-807a82d9eba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:33.300599 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:33.369129 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1395/2795] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:33.381553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bf225-a242-46a6-b5c3-3b96e8c92f16 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:33.383582 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-098bf225-a242-46a6-b5c3-3b96e8c92f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:33.445777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bf225-a242-46a6-b5c3-3b96e8c92f16 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:33.449602 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a78a83-6291-42e1-9273-fed3c3f186c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:33.450192 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a78a83-6291-42e1-9273-fed3c3f186c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:33.450472 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a78a83-6291-42e1-9273-fed3c3f186c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:33.450749 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a78a83-6291-42e1-9273-fed3c3f186c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:33.459717 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a78a83-6291-42e1-9273-fed3c3f186c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:33.461200 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a78a83-6291-42e1-9273-fed3c3f186c8 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:33.465081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:33.471212 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1401/2796] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:03:33.579746 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf784aa-add0-4637-b5c1-807a82d9eba4 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:33.589478 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf784aa-add0-4637-b5c1-807a82d9eba4 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:33.593828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e2712-562a-47e0-8a46-517165dc233a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:33.594411 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e2712-562a-47e0-8a46-517165dc233a service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:33.594692 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e2712-562a-47e0-8a46-517165dc233a service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:33.594966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e2712-562a-47e0-8a46-517165dc233a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:33.653291 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e2712-562a-47e0-8a46-517165dc233a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:33.655443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e2712-562a-47e0-8a46-517165dc233a service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:33.664630 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1396/2797] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:03:37.883265 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a78a83-6291-42e1-9273-fed3c3f186c8 service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:37.948736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a78a83-6291-42e1-9273-fed3c3f186c8 service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:37.953725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13c340-343c-4f54-b892-70077fe510b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:37.954342 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13c340-343c-4f54-b892-70077fe510b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:37.954626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13c340-343c-4f54-b892-70077fe510b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:37.954902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13c340-343c-4f54-b892-70077fe510b9 admin admin] PATH_INFO: `/v3/users/2450c53f5e694a21b0fdaf5d8fc15834` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:37.956210 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13c340-343c-4f54-b892-70077fe510b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2450c53f5e694a21b0fdaf5d8fc15834)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:37.958196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13c340-343c-4f54-b892-70077fe510b9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:38.144804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1a13c340-343c-4f54-b892-70077fe510b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2450c53f5e694a21b0fdaf5d8fc15834'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:38.157602 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1a13c340-343c-4f54-b892-70077fe510b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2450c53f5e694a21b0fdaf5d8fc15834'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:38.164485 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1a13c340-343c-4f54-b892-70077fe510b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2450c53f5e694a21b0fdaf5d8fc15834'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:38.216026 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1a13c340-343c-4f54-b892-70077fe510b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2450c53f5e694a21b0fdaf5d8fc15834'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:38.216653 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1a13c340-343c-4f54-b892-70077fe510b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2450c53f5e694a21b0fdaf5d8fc15834'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:38.223591 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1402/2798] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:03:37 2022] DELETE /identity/v3/users/2450c53f5e694a21b0fdaf5d8fc15834 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:38.228282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6d819-f755-43ce-8efd-ed47c6687f20 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:38.228740 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6d819-f755-43ce-8efd-ed47c6687f20 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:38.228857 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6d819-f755-43ce-8efd-ed47c6687f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:38.248646 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-00b6d819-f755-43ce-8efd-ed47c6687f20 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:38.249893 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-00b6d819-f755-43ce-8efd-ed47c6687f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:38.309122 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1397/2799] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:38.319018 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13c340-343c-4f54-b892-70077fe510b9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:38.320469 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-1a13c340-343c-4f54-b892-70077fe510b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:38.378030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13c340-343c-4f54-b892-70077fe510b9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:38.381667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da784785-6358-4382-b37f-480159b8cd05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:38.382267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da784785-6358-4382-b37f-480159b8cd05 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:38.382550 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da784785-6358-4382-b37f-480159b8cd05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:38.382828 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da784785-6358-4382-b37f-480159b8cd05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:38.393021 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da784785-6358-4382-b37f-480159b8cd05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:38.394469 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da784785-6358-4382-b37f-480159b8cd05 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:38.406145 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1403/2800] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:03:38.806279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6d819-f755-43ce-8efd-ed47c6687f20 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:38.807845 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-00b6d819-f755-43ce-8efd-ed47c6687f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:38.820614 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6d819-f755-43ce-8efd-ed47c6687f20 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:38.825236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7254a-037f-43c1-947c-fe248dda0dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:38.825906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7254a-037f-43c1-947c-fe248dda0dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:38.826458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7254a-037f-43c1-947c-fe248dda0dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:38.826749 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7254a-037f-43c1-947c-fe248dda0dd8 admin admin] PATH_INFO: `/v3/projects/d37bc6402ef64c3fb1630e8122487a6f` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:38.827942 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7254a-037f-43c1-947c-fe248dda0dd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d37bc6402ef64c3fb1630e8122487a6f)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:38.831310 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7254a-037f-43c1-947c-fe248dda0dd8 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:38.932926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:38.942414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-77d7254a-037f-43c1-947c-fe248dda0dd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd37bc6402ef64c3fb1630e8122487a6f', 'request_id': 'req-77d7254a-037f-43c1-947c-fe248dda0dd8'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:38.947692 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-77d7254a-037f-43c1-947c-fe248dda0dd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd37bc6402ef64c3fb1630e8122487a6f', 'request_id': 'req-77d7254a-037f-43c1-947c-fe248dda0dd8'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:39.022031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-77d7254a-037f-43c1-947c-fe248dda0dd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd37bc6402ef64c3fb1630e8122487a6f', 'request_id': 'req-77d7254a-037f-43c1-947c-fe248dda0dd8'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:39.034439 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-77d7254a-037f-43c1-947c-fe248dda0dd8 admin admin] The token cache is being invalidate because project d37bc6402ef64c3fb1630e8122487a6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:03:39.034728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-77d7254a-037f-43c1-947c-fe248dda0dd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:39.035384 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-77d7254a-037f-43c1-947c-fe248dda0dd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:39.036933 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1398/2801] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:03:38 2022] DELETE /identity/v3/projects/d37bc6402ef64c3fb1630e8122487a6f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:39.041295 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da784785-6358-4382-b37f-480159b8cd05 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:39.051539 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da784785-6358-4382-b37f-480159b8cd05 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:39.068312 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:39.108959 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2947de-cb7a-49b3-9b4f-d641b2b88b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:39.109598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2947de-cb7a-49b3-9b4f-d641b2b88b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:39.109882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2947de-cb7a-49b3-9b4f-d641b2b88b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:39.110685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2947de-cb7a-49b3-9b4f-d641b2b88b46 admin admin] PATH_INFO: `/v3/users/424554b896ac42ddb7625682405da5c9` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:39.115530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2947de-cb7a-49b3-9b4f-d641b2b88b46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=424554b896ac42ddb7625682405da5c9)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:39.115530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2947de-cb7a-49b3-9b4f-d641b2b88b46 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:39.332638 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8c2947de-cb7a-49b3-9b4f-d641b2b88b46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '424554b896ac42ddb7625682405da5c9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:39.344869 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8c2947de-cb7a-49b3-9b4f-d641b2b88b46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '424554b896ac42ddb7625682405da5c9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:39.352037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8c2947de-cb7a-49b3-9b4f-d641b2b88b46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '424554b896ac42ddb7625682405da5c9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:39.410408 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8c2947de-cb7a-49b3-9b4f-d641b2b88b46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '424554b896ac42ddb7625682405da5c9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:39.410927 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-8c2947de-cb7a-49b3-9b4f-d641b2b88b46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '424554b896ac42ddb7625682405da5c9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:39.417501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1404/2802] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:03:39 2022] DELETE /identity/v3/users/424554b896ac42ddb7625682405da5c9 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:39.801904 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7254a-037f-43c1-947c-fe248dda0dd8 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:39.806868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-77d7254a-037f-43c1-947c-fe248dda0dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:39.822990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7254a-037f-43c1-947c-fe248dda0dd8 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:39.893863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de1db2-402b-4fdc-9e55-f76d5672c9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:39.894776 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de1db2-402b-4fdc-9e55-f76d5672c9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:39.895268 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de1db2-402b-4fdc-9e55-f76d5672c9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:39.895692 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de1db2-402b-4fdc-9e55-f76d5672c9d0 admin admin] PATH_INFO: `/v3/projects/033d41adb04748a49de907282c56b877` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:39.897853 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de1db2-402b-4fdc-9e55-f76d5672c9d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=033d41adb04748a49de907282c56b877)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:39.899863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de1db2-402b-4fdc-9e55-f76d5672c9d0 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:40.000267 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-42de1db2-402b-4fdc-9e55-f76d5672c9d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '033d41adb04748a49de907282c56b877', 'request_id': 'req-42de1db2-402b-4fdc-9e55-f76d5672c9d0'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:40.006060 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-42de1db2-402b-4fdc-9e55-f76d5672c9d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '033d41adb04748a49de907282c56b877', 'request_id': 'req-42de1db2-402b-4fdc-9e55-f76d5672c9d0'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:40.067950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-42de1db2-402b-4fdc-9e55-f76d5672c9d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '033d41adb04748a49de907282c56b877', 'request_id': 'req-42de1db2-402b-4fdc-9e55-f76d5672c9d0'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:40.080963 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-42de1db2-402b-4fdc-9e55-f76d5672c9d0 admin admin] The token cache is being invalidate because project 033d41adb04748a49de907282c56b877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:03:40.081258 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-42de1db2-402b-4fdc-9e55-f76d5672c9d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:40.081914 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-42de1db2-402b-4fdc-9e55-f76d5672c9d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:40.086100 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1399/2803] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:03:39 2022] DELETE /identity/v3/projects/033d41adb04748a49de907282c56b877 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:40.087706 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2947de-cb7a-49b3-9b4f-d641b2b88b46 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:40.089209 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8c2947de-cb7a-49b3-9b4f-d641b2b88b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:40.102852 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2947de-cb7a-49b3-9b4f-d641b2b88b46 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:40.175121 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251d68f-41c3-456c-b0cd-da4775f78c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:40.175818 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251d68f-41c3-456c-b0cd-da4775f78c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:40.176102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251d68f-41c3-456c-b0cd-da4775f78c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:40.181044 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251d68f-41c3-456c-b0cd-da4775f78c65 admin admin] PATH_INFO: `/v3/users/2252eb6e54a044c5ad27b02bfbccc1f7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:40.181044 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251d68f-41c3-456c-b0cd-da4775f78c65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2252eb6e54a044c5ad27b02bfbccc1f7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:40.181044 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251d68f-41c3-456c-b0cd-da4775f78c65 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:40.465877 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:40.472332 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9251d68f-41c3-456c-b0cd-da4775f78c65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2252eb6e54a044c5ad27b02bfbccc1f7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:40.484450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9251d68f-41c3-456c-b0cd-da4775f78c65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2252eb6e54a044c5ad27b02bfbccc1f7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:40.492048 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9251d68f-41c3-456c-b0cd-da4775f78c65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2252eb6e54a044c5ad27b02bfbccc1f7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:40.541517 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9251d68f-41c3-456c-b0cd-da4775f78c65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2252eb6e54a044c5ad27b02bfbccc1f7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:40.542146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9251d68f-41c3-456c-b0cd-da4775f78c65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2252eb6e54a044c5ad27b02bfbccc1f7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:40.550589 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1405/2804] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:03:40 2022] DELETE /identity/v3/users/2252eb6e54a044c5ad27b02bfbccc1f7 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:40.949042 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de1db2-402b-4fdc-9e55-f76d5672c9d0 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:40.950544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-42de1db2-402b-4fdc-9e55-f76d5672c9d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:40.965169 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de1db2-402b-4fdc-9e55-f76d5672c9d0 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:41.015071 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:41.023997 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbb359-f2e8-4c8a-8105-b718d55020de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:41.024675 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbb359-f2e8-4c8a-8105-b718d55020de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:41.024954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbb359-f2e8-4c8a-8105-b718d55020de admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:41.025225 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbb359-f2e8-4c8a-8105-b718d55020de admin admin] PATH_INFO: `/v3/projects/3415db307a2c43cab0fab4ab3fc71347` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:41.029081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbb359-f2e8-4c8a-8105-b718d55020de admin admin] RBAC: Authorizing `identity:delete_project(project_id=3415db307a2c43cab0fab4ab3fc71347)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:41.029081 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbb359-f2e8-4c8a-8105-b718d55020de admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:41.160431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fedbb359-f2e8-4c8a-8105-b718d55020de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3415db307a2c43cab0fab4ab3fc71347', 'request_id': 'req-fedbb359-f2e8-4c8a-8105-b718d55020de'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:41.167791 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fedbb359-f2e8-4c8a-8105-b718d55020de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3415db307a2c43cab0fab4ab3fc71347', 'request_id': 'req-fedbb359-f2e8-4c8a-8105-b718d55020de'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:41.244399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fedbb359-f2e8-4c8a-8105-b718d55020de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3415db307a2c43cab0fab4ab3fc71347', 'request_id': 'req-fedbb359-f2e8-4c8a-8105-b718d55020de'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:41.257232 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fedbb359-f2e8-4c8a-8105-b718d55020de admin admin] The token cache is being invalidate because project 3415db307a2c43cab0fab4ab3fc71347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:03:41.257520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fedbb359-f2e8-4c8a-8105-b718d55020de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:41.258158 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fedbb359-f2e8-4c8a-8105-b718d55020de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:41.259730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1400/2805] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:03:40 2022] DELETE /identity/v3/projects/3415db307a2c43cab0fab4ab3fc71347 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:41.265795 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08246e02-e305-48ab-a32d-51b9ae4121b8 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:41.266287 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08246e02-e305-48ab-a32d-51b9ae4121b8 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:41.266416 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08246e02-e305-48ab-a32d-51b9ae4121b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:41.288341 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-08246e02-e305-48ab-a32d-51b9ae4121b8 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:41.289561 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-08246e02-e305-48ab-a32d-51b9ae4121b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:41.349185 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1406/2806] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:41.357447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086318e-c676-43b6-9936-3cccc9286797 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:41.357733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086318e-c676-43b6-9936-3cccc9286797 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:41.358006 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086318e-c676-43b6-9936-3cccc9286797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:41.379866 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-a086318e-c676-43b6-9936-3cccc9286797 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:41.381070 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a086318e-c676-43b6-9936-3cccc9286797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:41.389116 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1401/2807] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:41.394280 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08246e02-e305-48ab-a32d-51b9ae4121b8 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:41.404536 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08246e02-e305-48ab-a32d-51b9ae4121b8 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:41.409863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f68c0-8c44-4049-8d1f-9ef7e5062a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:41.410479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f68c0-8c44-4049-8d1f-9ef7e5062a99 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:41.410813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f68c0-8c44-4049-8d1f-9ef7e5062a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:41.411053 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f68c0-8c44-4049-8d1f-9ef7e5062a99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:41.413618 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f68c0-8c44-4049-8d1f-9ef7e5062a99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:03:41.413618 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f68c0-8c44-4049-8d1f-9ef7e5062a99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:41.413618 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f68c0-8c44-4049-8d1f-9ef7e5062a99 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:41.426148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1407/2808] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:03:41.430717 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086318e-c676-43b6-9936-3cccc9286797 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:41.440970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086318e-c676-43b6-9936-3cccc9286797 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:41.445757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5038e-f468-4b81-8cc6-dd6d70cc8df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:41.446395 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5038e-f468-4b81-8cc6-dd6d70cc8df4 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:41.446664 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5038e-f468-4b81-8cc6-dd6d70cc8df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:41.446937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5038e-f468-4b81-8cc6-dd6d70cc8df4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:41.447612 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5038e-f468-4b81-8cc6-dd6d70cc8df4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:41.450785 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5038e-f468-4b81-8cc6-dd6d70cc8df4 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:41.560864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1402/2809] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:03:41 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:41.565665 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f68c0-8c44-4049-8d1f-9ef7e5062a99 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:41.575901 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f68c0-8c44-4049-8d1f-9ef7e5062a99 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:41.633514 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd02883-c4e0-4750-81be-f47ed8c73d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:41.634129 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd02883-c4e0-4750-81be-f47ed8c73d99 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:41.634412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd02883-c4e0-4750-81be-f47ed8c73d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:41.634691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd02883-c4e0-4750-81be-f47ed8c73d99 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:41.635369 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd02883-c4e0-4750-81be-f47ed8c73d99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:41.637311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd02883-c4e0-4750-81be-f47ed8c73d99 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:41.725471 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1408/2810] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:03:41 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:41.729643 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5038e-f468-4b81-8cc6-dd6d70cc8df4 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:41.741328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5038e-f468-4b81-8cc6-dd6d70cc8df4 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:41.746114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a8ec4-5204-48d3-95c9-8e0c6ad13fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:41.746723 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a8ec4-5204-48d3-95c9-8e0c6ad13fdb admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:41.747004 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a8ec4-5204-48d3-95c9-8e0c6ad13fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:41.747315 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a8ec4-5204-48d3-95c9-8e0c6ad13fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:41.747959 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a8ec4-5204-48d3-95c9-8e0c6ad13fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:41.749553 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a8ec4-5204-48d3-95c9-8e0c6ad13fdb admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:41.759956 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1403/2811] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:41.764194 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd02883-c4e0-4750-81be-f47ed8c73d99 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:41.774696 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd02883-c4e0-4750-81be-f47ed8c73d99 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:41.779341 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de347fae-92d4-4bfb-9a10-f76bc85be92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:41.779935 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de347fae-92d4-4bfb-9a10-f76bc85be92c admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:41.780217 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de347fae-92d4-4bfb-9a10-f76bc85be92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:41.780488 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de347fae-92d4-4bfb-9a10-f76bc85be92c admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:41.781124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de347fae-92d4-4bfb-9a10-f76bc85be92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:41.783320 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de347fae-92d4-4bfb-9a10-f76bc85be92c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:41.793945 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1409/2812] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:41.797993 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a8ec4-5204-48d3-95c9-8e0c6ad13fdb admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:41.808623 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a8ec4-5204-48d3-95c9-8e0c6ad13fdb admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:41.813292 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403bf348-eb64-49cf-9161-4dca9d6f0b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:41.813887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403bf348-eb64-49cf-9161-4dca9d6f0b5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:41.814170 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403bf348-eb64-49cf-9161-4dca9d6f0b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:41.814448 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403bf348-eb64-49cf-9161-4dca9d6f0b5c admin admin] PATH_INFO: `/v3/projects/9a9d089b51804e7b905052d6d933475a/users/af56dae137f24de3819cbf4bab1c62d1/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:41.828702 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403bf348-eb64-49cf-9161-4dca9d6f0b5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a9d089b51804e7b905052d6d933475a, user_id=af56dae137f24de3819cbf4bab1c62d1, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:41.831400 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403bf348-eb64-49cf-9161-4dca9d6f0b5c admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:41.942932 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1404/2813] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:03:41 2022] PUT /identity/v3/projects/9a9d089b51804e7b905052d6d933475a/users/af56dae137f24de3819cbf4bab1c62d1/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:41.956121 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ede6bf-b75f-4718-9a95-63dd44cae373 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:41.956504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ede6bf-b75f-4718-9a95-63dd44cae373 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:41.956845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ede6bf-b75f-4718-9a95-63dd44cae373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:41.987745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-13ede6bf-b75f-4718-9a95-63dd44cae373 None None] MFA Rules not processed for user `af56dae137f24de3819cbf4bab1c62d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:41.988977 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-13ede6bf-b75f-4718-9a95-63dd44cae373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:42.000859 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:42.060117 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1410/2814] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:42.064667 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403bf348-eb64-49cf-9161-4dca9d6f0b5c admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:42.132667 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403bf348-eb64-49cf-9161-4dca9d6f0b5c admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:42.139786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca2955-d78d-419f-a77f-8964ffe6a39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:42.140427 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca2955-d78d-419f-a77f-8964ffe6a39e admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:42.140704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca2955-d78d-419f-a77f-8964ffe6a39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:42.141021 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca2955-d78d-419f-a77f-8964ffe6a39e admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:42.141672 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca2955-d78d-419f-a77f-8964ffe6a39e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:42.143958 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca2955-d78d-419f-a77f-8964ffe6a39e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:42.395936 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1405/2815] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:03:42 2022] POST /identity/v3/projects => generated 385 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:42.399044 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ede6bf-b75f-4718-9a95-63dd44cae373 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:42.410525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ede6bf-b75f-4718-9a95-63dd44cae373 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:42.468215 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900d195-1a1f-4d0b-8a57-2e5c4bd814b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:42.468816 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900d195-1a1f-4d0b-8a57-2e5c4bd814b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:42.469092 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900d195-1a1f-4d0b-8a57-2e5c4bd814b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:42.469361 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900d195-1a1f-4d0b-8a57-2e5c4bd814b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:42.469978 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900d195-1a1f-4d0b-8a57-2e5c4bd814b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:42.472464 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900d195-1a1f-4d0b-8a57-2e5c4bd814b9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:42.607909 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1411/2816] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:03:42 2022] POST /identity/v3/users => generated 299 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:42.609048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca2955-d78d-419f-a77f-8964ffe6a39e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:42.616241 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:42.620026 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca2955-d78d-419f-a77f-8964ffe6a39e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:42.624808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015792b-22d2-4a77-9548-247f817ced43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:42.629653 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015792b-22d2-4a77-9548-247f817ced43 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:42.629653 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015792b-22d2-4a77-9548-247f817ced43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:42.629653 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015792b-22d2-4a77-9548-247f817ced43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:42.629653 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015792b-22d2-4a77-9548-247f817ced43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:42.629653 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015792b-22d2-4a77-9548-247f817ced43 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:42.642965 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1406/2817] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:42.644101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900d195-1a1f-4d0b-8a57-2e5c4bd814b9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:42.654651 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900d195-1a1f-4d0b-8a57-2e5c4bd814b9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:42.659675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8dfd7e-d34a-4dd1-9ce9-09e8173aaceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:42.660352 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8dfd7e-d34a-4dd1-9ce9-09e8173aaceb admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:42.660644 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8dfd7e-d34a-4dd1-9ce9-09e8173aaceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:42.660922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8dfd7e-d34a-4dd1-9ce9-09e8173aaceb admin admin] PATH_INFO: `/v3/projects/579d0e3a685f493fa16ca2a52e926842/users/70de28f3f6ad4d09ac152b8ab6254d21/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:42.678095 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8dfd7e-d34a-4dd1-9ce9-09e8173aaceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=579d0e3a685f493fa16ca2a52e926842, user_id=70de28f3f6ad4d09ac152b8ab6254d21, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:42.680904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8dfd7e-d34a-4dd1-9ce9-09e8173aaceb admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:42.775506 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1412/2818] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:03:42 2022] PUT /identity/v3/projects/579d0e3a685f493fa16ca2a52e926842/users/70de28f3f6ad4d09ac152b8ab6254d21/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:42.779622 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015792b-22d2-4a77-9548-247f817ced43 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:42.790510 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015792b-22d2-4a77-9548-247f817ced43 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:42.847067 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8b1e1-2691-4e52-8280-4045febed31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:42.847725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8b1e1-2691-4e52-8280-4045febed31e admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:42.847990 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8b1e1-2691-4e52-8280-4045febed31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:42.848264 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8b1e1-2691-4e52-8280-4045febed31e admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:42.848904 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8b1e1-2691-4e52-8280-4045febed31e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:42.850624 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8b1e1-2691-4e52-8280-4045febed31e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:42.861022 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1407/2819] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:42.865333 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8dfd7e-d34a-4dd1-9ce9-09e8173aaceb admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:42.875990 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8dfd7e-d34a-4dd1-9ce9-09e8173aaceb admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:42.880863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74481633-6ced-47e2-8e88-8ad41f545fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:42.881535 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74481633-6ced-47e2-8e88-8ad41f545fe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:42.881815 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74481633-6ced-47e2-8e88-8ad41f545fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:42.882091 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74481633-6ced-47e2-8e88-8ad41f545fe0 admin admin] PATH_INFO: `/v3/projects/579d0e3a685f493fa16ca2a52e926842/users/70de28f3f6ad4d09ac152b8ab6254d21/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:42.884076 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74481633-6ced-47e2-8e88-8ad41f545fe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=579d0e3a685f493fa16ca2a52e926842, user_id=70de28f3f6ad4d09ac152b8ab6254d21, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:42.886711 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74481633-6ced-47e2-8e88-8ad41f545fe0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:42.898267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1413/2820] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:03:42 2022] PUT /identity/v3/projects/579d0e3a685f493fa16ca2a52e926842/users/70de28f3f6ad4d09ac152b8ab6254d21/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:42.909432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b152b8-7cbb-4e2f-8d73-eb305173d1a4 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:42.909684 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b152b8-7cbb-4e2f-8d73-eb305173d1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:42.909906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b152b8-7cbb-4e2f-8d73-eb305173d1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:42.937502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-82b152b8-7cbb-4e2f-8d73-eb305173d1a4 None None] MFA Rules not processed for user `70de28f3f6ad4d09ac152b8ab6254d21`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:42.938718 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-82b152b8-7cbb-4e2f-8d73-eb305173d1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:43.007408 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1408/2821] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:43.018982 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74481633-6ced-47e2-8e88-8ad41f545fe0 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:43.020716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-74481633-6ced-47e2-8e88-8ad41f545fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:43.078247 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74481633-6ced-47e2-8e88-8ad41f545fe0 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:43.082808 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29641fa-92b1-41b6-9092-543c0a97adc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:43.083438 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29641fa-92b1-41b6-9092-543c0a97adc4 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:43.083717 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29641fa-92b1-41b6-9092-543c0a97adc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:43.083986 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29641fa-92b1-41b6-9092-543c0a97adc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:43.092971 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29641fa-92b1-41b6-9092-543c0a97adc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:43.094456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29641fa-92b1-41b6-9092-543c0a97adc4 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:43.104623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1414/2822] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:03:44.373962 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b152b8-7cbb-4e2f-8d73-eb305173d1a4 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:44.437748 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b152b8-7cbb-4e2f-8d73-eb305173d1a4 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:44.442535 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a48de-d800-4f12-b72f-cd5cebd37fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:44.443269 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a48de-d800-4f12-b72f-cd5cebd37fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:44.443551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a48de-d800-4f12-b72f-cd5cebd37fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:44.443824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a48de-d800-4f12-b72f-cd5cebd37fb6 admin admin] PATH_INFO: `/v3/users/af56dae137f24de3819cbf4bab1c62d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:44.444954 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a48de-d800-4f12-b72f-cd5cebd37fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af56dae137f24de3819cbf4bab1c62d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:44.446904 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a48de-d800-4f12-b72f-cd5cebd37fb6 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:44.464706 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:44.677321 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1f2a48de-d800-4f12-b72f-cd5cebd37fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af56dae137f24de3819cbf4bab1c62d1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:44.690518 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1f2a48de-d800-4f12-b72f-cd5cebd37fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af56dae137f24de3819cbf4bab1c62d1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:44.698414 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1f2a48de-d800-4f12-b72f-cd5cebd37fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af56dae137f24de3819cbf4bab1c62d1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:44.781518 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1f2a48de-d800-4f12-b72f-cd5cebd37fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af56dae137f24de3819cbf4bab1c62d1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:44.782149 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-1f2a48de-d800-4f12-b72f-cd5cebd37fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af56dae137f24de3819cbf4bab1c62d1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:44.788891 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1409/2823] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:03:44 2022] DELETE /identity/v3/users/af56dae137f24de3819cbf4bab1c62d1 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:44.793422 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab054fdd-c856-43e0-b0de-7577a0bfbe44 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:44.793905 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab054fdd-c856-43e0-b0de-7577a0bfbe44 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:44.794022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab054fdd-c856-43e0-b0de-7577a0bfbe44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:44.813159 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ab054fdd-c856-43e0-b0de-7577a0bfbe44 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:44.815107 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ab054fdd-c856-43e0-b0de-7577a0bfbe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:44.819279 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:44.875031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1415/2824] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:44.885609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a48de-d800-4f12-b72f-cd5cebd37fb6 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:44.902044 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-1f2a48de-d800-4f12-b72f-cd5cebd37fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:44.944937 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a48de-d800-4f12-b72f-cd5cebd37fb6 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:44.948683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269530bd-508a-4715-94e9-4c00ec74f14b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:44.949287 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269530bd-508a-4715-94e9-4c00ec74f14b service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:44.949566 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269530bd-508a-4715-94e9-4c00ec74f14b service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:44.949838 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269530bd-508a-4715-94e9-4c00ec74f14b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:44.961013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269530bd-508a-4715-94e9-4c00ec74f14b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:44.962458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269530bd-508a-4715-94e9-4c00ec74f14b service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:44.972415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1410/2825] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:03:45.343963 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab054fdd-c856-43e0-b0de-7577a0bfbe44 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:45.345504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ab054fdd-c856-43e0-b0de-7577a0bfbe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:45.360708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab054fdd-c856-43e0-b0de-7577a0bfbe44 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:45.366616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:45.369112 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:45.369112 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:45.369112 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9 admin admin] PATH_INFO: `/v3/projects/9a9d089b51804e7b905052d6d933475a` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:45.369529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a9d089b51804e7b905052d6d933475a)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:45.371744 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:45.503436 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a9d089b51804e7b905052d6d933475a', 'request_id': 'req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:45.569899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a9d089b51804e7b905052d6d933475a', 'request_id': 'req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:45.577272 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a9d089b51804e7b905052d6d933475a', 'request_id': 'req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:45.588524 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9 admin admin] The token cache is being invalidate because project 9a9d089b51804e7b905052d6d933475a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:03:45.588813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:45.589470 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:45.590970 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1416/2826] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:03:45 2022] DELETE /identity/v3/projects/9a9d089b51804e7b905052d6d933475a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:45.594997 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269530bd-508a-4715-94e9-4c00ec74f14b service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:45.605267 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269530bd-508a-4715-94e9-4c00ec74f14b service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:45.663105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90004222-de61-49a6-a9d0-9f0ec41267da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:45.663765 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90004222-de61-49a6-a9d0-9f0ec41267da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:45.664051 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90004222-de61-49a6-a9d0-9f0ec41267da admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:45.664321 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90004222-de61-49a6-a9d0-9f0ec41267da admin admin] PATH_INFO: `/v3/users/70de28f3f6ad4d09ac152b8ab6254d21` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:45.665444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90004222-de61-49a6-a9d0-9f0ec41267da admin admin] RBAC: Authorizing `identity:delete_user(user_id=70de28f3f6ad4d09ac152b8ab6254d21)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:45.667389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90004222-de61-49a6-a9d0-9f0ec41267da admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:45.881293 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:45.887293 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-90004222-de61-49a6-a9d0-9f0ec41267da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70de28f3f6ad4d09ac152b8ab6254d21'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:45.900114 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-90004222-de61-49a6-a9d0-9f0ec41267da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70de28f3f6ad4d09ac152b8ab6254d21'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:45.907006 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-90004222-de61-49a6-a9d0-9f0ec41267da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70de28f3f6ad4d09ac152b8ab6254d21'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:45.975406 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-90004222-de61-49a6-a9d0-9f0ec41267da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70de28f3f6ad4d09ac152b8ab6254d21'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:45.976120 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-90004222-de61-49a6-a9d0-9f0ec41267da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70de28f3f6ad4d09ac152b8ab6254d21'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:45.982227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1411/2827] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:03:45 2022] DELETE /identity/v3/users/70de28f3f6ad4d09ac152b8ab6254d21 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:46.464039 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:46.465564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:46.481415 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0311a70-a9ae-4e52-8b8a-9dd34753c3a9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:46.543423 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e74e0-dfae-464e-8c4f-6827e4d983fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:46.544110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e74e0-dfae-464e-8c4f-6827e4d983fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:46.544399 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e74e0-dfae-464e-8c4f-6827e4d983fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:46.544681 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e74e0-dfae-464e-8c4f-6827e4d983fd admin admin] PATH_INFO: `/v3/projects/579d0e3a685f493fa16ca2a52e926842` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:46.545885 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e74e0-dfae-464e-8c4f-6827e4d983fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=579d0e3a685f493fa16ca2a52e926842)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:46.547875 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e74e0-dfae-464e-8c4f-6827e4d983fd admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:46.647841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:46.658790 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-501e74e0-dfae-464e-8c4f-6827e4d983fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '579d0e3a685f493fa16ca2a52e926842', 'request_id': 'req-501e74e0-dfae-464e-8c4f-6827e4d983fd'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:46.741756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-501e74e0-dfae-464e-8c4f-6827e4d983fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '579d0e3a685f493fa16ca2a52e926842', 'request_id': 'req-501e74e0-dfae-464e-8c4f-6827e4d983fd'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:46.748740 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-501e74e0-dfae-464e-8c4f-6827e4d983fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '579d0e3a685f493fa16ca2a52e926842', 'request_id': 'req-501e74e0-dfae-464e-8c4f-6827e4d983fd'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:46.760255 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-501e74e0-dfae-464e-8c4f-6827e4d983fd admin admin] The token cache is being invalidate because project 579d0e3a685f493fa16ca2a52e926842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:03:46.760544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-501e74e0-dfae-464e-8c4f-6827e4d983fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:46.761201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-501e74e0-dfae-464e-8c4f-6827e4d983fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:03:46.762747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1417/2828] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:03:46 2022] DELETE /identity/v3/projects/579d0e3a685f493fa16ca2a52e926842 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:46.768300 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3b329-b443-4570-8998-88000119be31 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:46.768820 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3b329-b443-4570-8998-88000119be31 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:46.769015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3b329-b443-4570-8998-88000119be31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:46.789623 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-d4b3b329-b443-4570-8998-88000119be31 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:46.791006 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d4b3b329-b443-4570-8998-88000119be31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:46.852137 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1412/2829] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:46.861622 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab752c5e-9bfa-4452-b4ff-71f710965a79 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:46.861937 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab752c5e-9bfa-4452-b4ff-71f710965a79 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:46.862208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab752c5e-9bfa-4452-b4ff-71f710965a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:46.888341 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ab752c5e-9bfa-4452-b4ff-71f710965a79 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:46.889635 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ab752c5e-9bfa-4452-b4ff-71f710965a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:46.902129 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1418/2830] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:46.903636 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3b329-b443-4570-8998-88000119be31 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:46.913839 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3b329-b443-4570-8998-88000119be31 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:46.918510 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228c336-b25b-4227-be72-462f7974263b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:46.919124 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228c336-b25b-4227-be72-462f7974263b admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:46.919433 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228c336-b25b-4227-be72-462f7974263b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:46.919704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228c336-b25b-4227-be72-462f7974263b admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:46.920349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228c336-b25b-4227-be72-462f7974263b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:03:46.920652 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228c336-b25b-4227-be72-462f7974263b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:46.922243 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228c336-b25b-4227-be72-462f7974263b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:46.935295 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1413/2831] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:03:46.940757 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab752c5e-9bfa-4452-b4ff-71f710965a79 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:46.950140 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab752c5e-9bfa-4452-b4ff-71f710965a79 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:46.954848 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b363c-fd93-4e75-a873-a589756085e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:46.955478 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b363c-fd93-4e75-a873-a589756085e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:46.955761 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b363c-fd93-4e75-a873-a589756085e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:46.956034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b363c-fd93-4e75-a873-a589756085e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:46.956664 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b363c-fd93-4e75-a873-a589756085e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:46.958553 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b363c-fd93-4e75-a873-a589756085e8 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:47.107151 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1419/2832] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:03:46 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:47.108393 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228c336-b25b-4227-be72-462f7974263b admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:47.120048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228c336-b25b-4227-be72-462f7974263b admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:47.189191 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d4580-d14e-4575-a484-96d6b8196e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:47.189808 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d4580-d14e-4575-a484-96d6b8196e30 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:47.190086 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d4580-d14e-4575-a484-96d6b8196e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:47.190355 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d4580-d14e-4575-a484-96d6b8196e30 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:47.191108 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d4580-d14e-4575-a484-96d6b8196e30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:47.193546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d4580-d14e-4575-a484-96d6b8196e30 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:47.299815 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1414/2833] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:03:47 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:47.301091 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b363c-fd93-4e75-a873-a589756085e8 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:47.311829 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b363c-fd93-4e75-a873-a589756085e8 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:47.316669 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e256a74-c34f-4463-ac6e-80e0442817e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:47.317324 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e256a74-c34f-4463-ac6e-80e0442817e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:47.317605 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e256a74-c34f-4463-ac6e-80e0442817e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:47.317874 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e256a74-c34f-4463-ac6e-80e0442817e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:47.318543 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e256a74-c34f-4463-ac6e-80e0442817e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:47.320226 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e256a74-c34f-4463-ac6e-80e0442817e5 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:47.330930 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1420/2834] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:47.342806 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d4580-d14e-4575-a484-96d6b8196e30 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:47.353563 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d4580-d14e-4575-a484-96d6b8196e30 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:47.358324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e330f-7734-4653-923e-122eb20cd66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:47.358961 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e330f-7734-4653-923e-122eb20cd66a admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:47.359276 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e330f-7734-4653-923e-122eb20cd66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:47.359569 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e330f-7734-4653-923e-122eb20cd66a admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:47.360191 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e330f-7734-4653-923e-122eb20cd66a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:47.361837 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e330f-7734-4653-923e-122eb20cd66a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:47.376476 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1415/2835] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:47.381005 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e256a74-c34f-4463-ac6e-80e0442817e5 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:47.392960 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e256a74-c34f-4463-ac6e-80e0442817e5 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:47.397645 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440448d-e312-48f1-b9d6-512759359e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:47.398311 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440448d-e312-48f1-b9d6-512759359e0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:47.402767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440448d-e312-48f1-b9d6-512759359e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:47.402767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440448d-e312-48f1-b9d6-512759359e0b admin admin] PATH_INFO: `/v3/projects/059b5a5c11b84099b20e81b95106eb04/users/38337bff41f74974827b46795fe425fa/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:47.415599 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440448d-e312-48f1-b9d6-512759359e0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=059b5a5c11b84099b20e81b95106eb04, user_id=38337bff41f74974827b46795fe425fa, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:47.418294 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440448d-e312-48f1-b9d6-512759359e0b admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:47.519737 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1421/2836] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:03:47 2022] PUT /identity/v3/projects/059b5a5c11b84099b20e81b95106eb04/users/38337bff41f74974827b46795fe425fa/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:47.524824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eba893-895c-4068-8389-4c585d410a42 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:47.525107 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eba893-895c-4068-8389-4c585d410a42 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:47.525380 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eba893-895c-4068-8389-4c585d410a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:47.545011 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-c2eba893-895c-4068-8389-4c585d410a42 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:47.546144 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c2eba893-895c-4068-8389-4c585d410a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:47.564688 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:47.610247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1416/2837] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:47.620127 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440448d-e312-48f1-b9d6-512759359e0b admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:47.621833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f440448d-e312-48f1-b9d6-512759359e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:47.681516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440448d-e312-48f1-b9d6-512759359e0b admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:47.685471 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d98e4c-7494-475f-9cd2-aba971302e7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:47.686086 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d98e4c-7494-475f-9cd2-aba971302e7f service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:47.686364 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d98e4c-7494-475f-9cd2-aba971302e7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:47.686625 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d98e4c-7494-475f-9cd2-aba971302e7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:47.696461 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d98e4c-7494-475f-9cd2-aba971302e7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:47.698173 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d98e4c-7494-475f-9cd2-aba971302e7f service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:47.708615 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1422/2838] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:03:49.148986 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee89c40-6db7-44e9-afed-c01229ed8062 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:49.149516 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee89c40-6db7-44e9-afed-c01229ed8062 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:49.149516 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee89c40-6db7-44e9-afed-c01229ed8062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:49.178324 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-4ee89c40-6db7-44e9-afed-c01229ed8062 None None] MFA Rules not processed for user `38337bff41f74974827b46795fe425fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:49.180282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4ee89c40-6db7-44e9-afed-c01229ed8062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:49.249429 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1417/2839] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:49.253775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d98e4c-7494-475f-9cd2-aba971302e7f service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:49.263974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d98e4c-7494-475f-9cd2-aba971302e7f service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:49.269515 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d40035-0f4b-4976-815a-58f61953e8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:49.273595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d40035-0f4b-4976-815a-58f61953e8c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:49.273595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d40035-0f4b-4976-815a-58f61953e8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:49.273595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d40035-0f4b-4976-815a-58f61953e8c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:49.273595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d40035-0f4b-4976-815a-58f61953e8c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:49.273595 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d40035-0f4b-4976-815a-58f61953e8c1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:49.391742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1423/2840] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:03:49 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:49.393091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee89c40-6db7-44e9-afed-c01229ed8062 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:49.404482 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee89c40-6db7-44e9-afed-c01229ed8062 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:49.475441 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2d490-ee4e-4abc-b93f-a82c2612e596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:49.476152 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2d490-ee4e-4abc-b93f-a82c2612e596 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:49.476432 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2d490-ee4e-4abc-b93f-a82c2612e596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:49.476709 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2d490-ee4e-4abc-b93f-a82c2612e596 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:49.477347 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a2d490-ee4e-4abc-b93f-a82c2612e596 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:49.479294 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a2d490-ee4e-4abc-b93f-a82c2612e596 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:49.553183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1418/2841] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:03:49 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:03:49.557104 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d40035-0f4b-4976-815a-58f61953e8c1 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:49.568782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d40035-0f4b-4976-815a-58f61953e8c1 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:49.574124 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b252fe-124e-42e3-ad41-d3b51243a6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:49.574872 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b252fe-124e-42e3-ad41-d3b51243a6bb admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:49.575297 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b252fe-124e-42e3-ad41-d3b51243a6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:49.575830 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b252fe-124e-42e3-ad41-d3b51243a6bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:49.576619 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b252fe-124e-42e3-ad41-d3b51243a6bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:49.578284 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b252fe-124e-42e3-ad41-d3b51243a6bb admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:49.589594 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1424/2842] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:49.593864 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2d490-ee4e-4abc-b93f-a82c2612e596 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:49.604547 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2d490-ee4e-4abc-b93f-a82c2612e596 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:49.609180 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b671765-9da0-45b6-86e5-955065c8980b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:49.609845 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b671765-9da0-45b6-86e5-955065c8980b admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:49.610127 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b671765-9da0-45b6-86e5-955065c8980b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:49.610402 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b671765-9da0-45b6-86e5-955065c8980b admin admin] PATH_INFO: `/v3/projects/4cd9ac1a37a943f78e5d451efeb029ab/users/0fe8d572c4e84444950421e50a166acb/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:49.625077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b671765-9da0-45b6-86e5-955065c8980b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd9ac1a37a943f78e5d451efeb029ab, user_id=0fe8d572c4e84444950421e50a166acb, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:49.627809 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b671765-9da0-45b6-86e5-955065c8980b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:49.701215 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1419/2843] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:03:49 2022] PUT /identity/v3/projects/4cd9ac1a37a943f78e5d451efeb029ab/users/0fe8d572c4e84444950421e50a166acb/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:49.705764 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b252fe-124e-42e3-ad41-d3b51243a6bb admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:49.716682 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b252fe-124e-42e3-ad41-d3b51243a6bb admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:49.743385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:49.773725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690db9f-0d51-4052-853b-8ad7372e6f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:49.774341 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690db9f-0d51-4052-853b-8ad7372e6f2a admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:49.774626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690db9f-0d51-4052-853b-8ad7372e6f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:49.774907 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690db9f-0d51-4052-853b-8ad7372e6f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:49.775582 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690db9f-0d51-4052-853b-8ad7372e6f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:49.777181 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690db9f-0d51-4052-853b-8ad7372e6f2a admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:49.788158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1425/2844] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:03:49.792364 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b671765-9da0-45b6-86e5-955065c8980b admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:49.802759 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b671765-9da0-45b6-86e5-955065c8980b admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:49.807467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a227a-2264-486c-abef-0e14deebc4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:49.808113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a227a-2264-486c-abef-0e14deebc4c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:49.808394 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a227a-2264-486c-abef-0e14deebc4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:49.808675 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a227a-2264-486c-abef-0e14deebc4c2 admin admin] PATH_INFO: `/v3/projects/4cd9ac1a37a943f78e5d451efeb029ab/users/0fe8d572c4e84444950421e50a166acb/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:49.810585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a227a-2264-486c-abef-0e14deebc4c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd9ac1a37a943f78e5d451efeb029ab, user_id=0fe8d572c4e84444950421e50a166acb, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:49.813259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a227a-2264-486c-abef-0e14deebc4c2 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:49.823806 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1420/2845] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:03:49 2022] PUT /identity/v3/projects/4cd9ac1a37a943f78e5d451efeb029ab/users/0fe8d572c4e84444950421e50a166acb/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:03:51.174650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18046bd2-dfe3-4254-90c6-0c451cc68482 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:51.174650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18046bd2-dfe3-4254-90c6-0c451cc68482 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:51.174650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18046bd2-dfe3-4254-90c6-0c451cc68482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:51.203602 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-18046bd2-dfe3-4254-90c6-0c451cc68482 None None] MFA Rules not processed for user `0fe8d572c4e84444950421e50a166acb`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:51.208813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-18046bd2-dfe3-4254-90c6-0c451cc68482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:51.284568 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1426/2846] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:03:51.296586 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a227a-2264-486c-abef-0e14deebc4c2 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:51.298456 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b29a227a-2264-486c-abef-0e14deebc4c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:51.305419 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:03:51.360369 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a227a-2264-486c-abef-0e14deebc4c2 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:51.366159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a1f9a-1b30-465f-aa0b-b98772b16f16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:51.366159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a1f9a-1b30-465f-aa0b-b98772b16f16 service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:51.366159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a1f9a-1b30-465f-aa0b-b98772b16f16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:51.366159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a1f9a-1b30-465f-aa0b-b98772b16f16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:51.426854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a1f9a-1b30-465f-aa0b-b98772b16f16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:51.428266 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a1f9a-1b30-465f-aa0b-b98772b16f16 service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:51.437817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1421/2847] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:03:53.322746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18046bd2-dfe3-4254-90c6-0c451cc68482 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:03:53.324221 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-18046bd2-dfe3-4254-90c6-0c451cc68482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:53.336509 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18046bd2-dfe3-4254-90c6-0c451cc68482 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:03:53.340524 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42986805-90bc-4c1b-8b96-be3ca9df4a6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:03:53.344782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42986805-90bc-4c1b-8b96-be3ca9df4a6b service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:53.344782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42986805-90bc-4c1b-8b96-be3ca9df4a6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:53.344782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42986805-90bc-4c1b-8b96-be3ca9df4a6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:53.351455 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42986805-90bc-4c1b-8b96-be3ca9df4a6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:03:53.352808 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42986805-90bc-4c1b-8b96-be3ca9df4a6b service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:03:53.363277 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1427/2848] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:03:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:03:53.493577 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193c4ce-773b-4f1d-bb28-d81a86e13217 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:03:53.494084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193c4ce-773b-4f1d-bb28-d81a86e13217 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:03:53.499543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193c4ce-773b-4f1d-bb28-d81a86e13217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:03:53.516311 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-b193c4ce-773b-4f1d-bb28-d81a86e13217 None None] MFA Rules not processed for user `7483b0e52765411d8444dfdd46549c5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:03:53.517794 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b193c4ce-773b-4f1d-bb28-d81a86e13217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:03:53.579811 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1422/2849] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:03:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:04:03.358714 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=56 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:03.572901 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=71 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:09.352724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42986805-90bc-4c1b-8b96-be3ca9df4a6b service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:09.418590 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42986805-90bc-4c1b-8b96-be3ca9df4a6b service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:09.423202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e824d-4b02-4b26-88e4-9c1e3095e0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:09.424504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e824d-4b02-4b26-88e4-9c1e3095e0ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:09.424784 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e824d-4b02-4b26-88e4-9c1e3095e0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:09.425055 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e824d-4b02-4b26-88e4-9c1e3095e0ad admin admin] PATH_INFO: `/v3/users/38337bff41f74974827b46795fe425fa` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:09.426180 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e824d-4b02-4b26-88e4-9c1e3095e0ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=38337bff41f74974827b46795fe425fa)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:09.428136 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e824d-4b02-4b26-88e4-9c1e3095e0ad admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:09.606401 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b55e824d-4b02-4b26-88e4-9c1e3095e0ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38337bff41f74974827b46795fe425fa'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:09.618858 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b55e824d-4b02-4b26-88e4-9c1e3095e0ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38337bff41f74974827b46795fe425fa'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:09.625824 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b55e824d-4b02-4b26-88e4-9c1e3095e0ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38337bff41f74974827b46795fe425fa'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:09.700704 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b55e824d-4b02-4b26-88e4-9c1e3095e0ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38337bff41f74974827b46795fe425fa'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:09.701457 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b55e824d-4b02-4b26-88e4-9c1e3095e0ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38337bff41f74974827b46795fe425fa'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:09.707984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1428/2850] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:04:09 2022] DELETE /identity/v3/users/38337bff41f74974827b46795fe425fa => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:04:09.977063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193c4ce-773b-4f1d-bb28-d81a86e13217 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:09.990880 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193c4ce-773b-4f1d-bb28-d81a86e13217 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:10.051465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736abeba-ec32-4c7f-a01d-8ce110111425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:10.052105 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736abeba-ec32-4c7f-a01d-8ce110111425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:10.052395 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736abeba-ec32-4c7f-a01d-8ce110111425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:10.052667 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736abeba-ec32-4c7f-a01d-8ce110111425 admin admin] PATH_INFO: `/v3/projects/059b5a5c11b84099b20e81b95106eb04` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:10.054120 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736abeba-ec32-4c7f-a01d-8ce110111425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=059b5a5c11b84099b20e81b95106eb04)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:10.056333 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736abeba-ec32-4c7f-a01d-8ce110111425 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:10.148301 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-736abeba-ec32-4c7f-a01d-8ce110111425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '059b5a5c11b84099b20e81b95106eb04', 'request_id': 'req-736abeba-ec32-4c7f-a01d-8ce110111425'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:10.153882 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-736abeba-ec32-4c7f-a01d-8ce110111425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '059b5a5c11b84099b20e81b95106eb04', 'request_id': 'req-736abeba-ec32-4c7f-a01d-8ce110111425'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:10.197870 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-736abeba-ec32-4c7f-a01d-8ce110111425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '059b5a5c11b84099b20e81b95106eb04', 'request_id': 'req-736abeba-ec32-4c7f-a01d-8ce110111425'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:10.210326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-736abeba-ec32-4c7f-a01d-8ce110111425 admin admin] The token cache is being invalidate because project 059b5a5c11b84099b20e81b95106eb04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:04:10.210622 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-736abeba-ec32-4c7f-a01d-8ce110111425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:10.211307 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-736abeba-ec32-4c7f-a01d-8ce110111425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:10.212821 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1423/2851] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:04:09 2022] DELETE /identity/v3/projects/059b5a5c11b84099b20e81b95106eb04 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:04:10.217123 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e824d-4b02-4b26-88e4-9c1e3095e0ad admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:10.218551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-b55e824d-4b02-4b26-88e4-9c1e3095e0ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:10.238128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e824d-4b02-4b26-88e4-9c1e3095e0ad admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:10.288321 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060404e-4ae8-4704-80bc-0402ceb5ef63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:10.288965 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060404e-4ae8-4704-80bc-0402ceb5ef63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:10.289250 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060404e-4ae8-4704-80bc-0402ceb5ef63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:10.291231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060404e-4ae8-4704-80bc-0402ceb5ef63 admin admin] PATH_INFO: `/v3/users/0fe8d572c4e84444950421e50a166acb` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:10.292411 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060404e-4ae8-4704-80bc-0402ceb5ef63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fe8d572c4e84444950421e50a166acb)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:10.294668 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060404e-4ae8-4704-80bc-0402ceb5ef63 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:10.319771 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:10.470080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b060404e-4ae8-4704-80bc-0402ceb5ef63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fe8d572c4e84444950421e50a166acb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:10.482140 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b060404e-4ae8-4704-80bc-0402ceb5ef63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fe8d572c4e84444950421e50a166acb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:10.489081 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b060404e-4ae8-4704-80bc-0402ceb5ef63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fe8d572c4e84444950421e50a166acb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:10.540607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b060404e-4ae8-4704-80bc-0402ceb5ef63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fe8d572c4e84444950421e50a166acb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:10.541231 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-b060404e-4ae8-4704-80bc-0402ceb5ef63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fe8d572c4e84444950421e50a166acb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:10.547720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1429/2852] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:04:10 2022] DELETE /identity/v3/users/0fe8d572c4e84444950421e50a166acb => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:04:10.805713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736abeba-ec32-4c7f-a01d-8ce110111425 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:10.807305 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-736abeba-ec32-4c7f-a01d-8ce110111425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:10.823511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736abeba-ec32-4c7f-a01d-8ce110111425 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:10.881202 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936190e-dd36-4d89-b052-4cba96b21610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:10.881831 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936190e-dd36-4d89-b052-4cba96b21610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:10.882116 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936190e-dd36-4d89-b052-4cba96b21610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:10.882388 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936190e-dd36-4d89-b052-4cba96b21610 admin admin] PATH_INFO: `/v3/projects/4cd9ac1a37a943f78e5d451efeb029ab` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:10.883543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936190e-dd36-4d89-b052-4cba96b21610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd9ac1a37a943f78e5d451efeb029ab)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:10.885444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936190e-dd36-4d89-b052-4cba96b21610 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:10.975900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:10.988757 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-9936190e-dd36-4d89-b052-4cba96b21610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd9ac1a37a943f78e5d451efeb029ab', 'request_id': 'req-9936190e-dd36-4d89-b052-4cba96b21610'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:10.994680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-9936190e-dd36-4d89-b052-4cba96b21610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd9ac1a37a943f78e5d451efeb029ab', 'request_id': 'req-9936190e-dd36-4d89-b052-4cba96b21610'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:11.043995 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-9936190e-dd36-4d89-b052-4cba96b21610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd9ac1a37a943f78e5d451efeb029ab', 'request_id': 'req-9936190e-dd36-4d89-b052-4cba96b21610'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:11.063271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-9936190e-dd36-4d89-b052-4cba96b21610 admin admin] The token cache is being invalidate because project 4cd9ac1a37a943f78e5d451efeb029ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:04:11.063578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-9936190e-dd36-4d89-b052-4cba96b21610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:11.064316 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-9936190e-dd36-4d89-b052-4cba96b21610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:11.065857 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1424/2853] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:04:10 2022] DELETE /identity/v3/projects/4cd9ac1a37a943f78e5d451efeb029ab => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:04:11.070775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bff83-7246-486a-8625-49dcb9fc90ec None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:11.071146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bff83-7246-486a-8625-49dcb9fc90ec None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:11.071446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bff83-7246-486a-8625-49dcb9fc90ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:11.091607 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-823bff83-7246-486a-8625-49dcb9fc90ec None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:04:11.092918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-823bff83-7246-486a-8625-49dcb9fc90ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:11.153510 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1430/2854] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:04:11.162013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85b4bc-2d54-47e1-bcd9-566fc639b881 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:11.162302 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85b4bc-2d54-47e1-bcd9-566fc639b881 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:11.162580 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85b4bc-2d54-47e1-bcd9-566fc639b881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:11.183026 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-5a85b4bc-2d54-47e1-bcd9-566fc639b881 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:04:11.184272 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-5a85b4bc-2d54-47e1-bcd9-566fc639b881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:11.192502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1425/2855] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:04:11.202290 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bff83-7246-486a-8625-49dcb9fc90ec None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:11.212580 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bff83-7246-486a-8625-49dcb9fc90ec None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:11.217445 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7765617-207c-47ed-a9f0-004c91fcf6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:11.218063 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7765617-207c-47ed-a9f0-004c91fcf6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:11.218346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7765617-207c-47ed-a9f0-004c91fcf6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:11.218623 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7765617-207c-47ed-a9f0-004c91fcf6e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:11.219487 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7765617-207c-47ed-a9f0-004c91fcf6e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:04:11.219739 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7765617-207c-47ed-a9f0-004c91fcf6e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:11.221862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7765617-207c-47ed-a9f0-004c91fcf6e9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:11.233964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1431/2856] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:04:11.239119 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85b4bc-2d54-47e1-bcd9-566fc639b881 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:11.248568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85b4bc-2d54-47e1-bcd9-566fc639b881 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:11.253369 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd30cc-3448-4f35-a8f2-9bcba49604ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:11.254092 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd30cc-3448-4f35-a8f2-9bcba49604ac admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:11.254373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd30cc-3448-4f35-a8f2-9bcba49604ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:11.254650 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd30cc-3448-4f35-a8f2-9bcba49604ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:11.255326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd30cc-3448-4f35-a8f2-9bcba49604ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:11.257229 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd30cc-3448-4f35-a8f2-9bcba49604ac admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:11.363002 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1426/2857] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:04:11 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:04:11.366673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7765617-207c-47ed-a9f0-004c91fcf6e9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:11.380384 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7765617-207c-47ed-a9f0-004c91fcf6e9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:11.451943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce56461-b4fd-4ae0-b15c-72c520b27163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:11.452556 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce56461-b4fd-4ae0-b15c-72c520b27163 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:11.452841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce56461-b4fd-4ae0-b15c-72c520b27163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:11.453118 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce56461-b4fd-4ae0-b15c-72c520b27163 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:11.453826 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce56461-b4fd-4ae0-b15c-72c520b27163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:11.456303 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce56461-b4fd-4ae0-b15c-72c520b27163 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:11.596106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1432/2858] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:04:11 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:04:11.598974 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd30cc-3448-4f35-a8f2-9bcba49604ac admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:11.612101 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd30cc-3448-4f35-a8f2-9bcba49604ac admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:11.616829 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62295bf2-ef98-4c9f-afeb-f3bf8276fcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:11.617356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62295bf2-ef98-4c9f-afeb-f3bf8276fcee admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:11.617633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62295bf2-ef98-4c9f-afeb-f3bf8276fcee admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:11.617904 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62295bf2-ef98-4c9f-afeb-f3bf8276fcee admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:11.618575 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62295bf2-ef98-4c9f-afeb-f3bf8276fcee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:11.620244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62295bf2-ef98-4c9f-afeb-f3bf8276fcee admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:11.630849 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1427/2859] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:04:11.640745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce56461-b4fd-4ae0-b15c-72c520b27163 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:11.651630 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce56461-b4fd-4ae0-b15c-72c520b27163 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:11.656570 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771696a3-d731-464a-82ed-517cc774babf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:11.657158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771696a3-d731-464a-82ed-517cc774babf admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:11.657441 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771696a3-d731-464a-82ed-517cc774babf admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:11.657715 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771696a3-d731-464a-82ed-517cc774babf admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:11.658912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771696a3-d731-464a-82ed-517cc774babf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:11.660547 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771696a3-d731-464a-82ed-517cc774babf admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:11.671212 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1433/2860] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:04:11.679913 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62295bf2-ef98-4c9f-afeb-f3bf8276fcee admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:11.690345 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62295bf2-ef98-4c9f-afeb-f3bf8276fcee admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:11.694997 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6f339-e42a-4e74-98ae-07c51cff4431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:11.695678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6f339-e42a-4e74-98ae-07c51cff4431 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:11.695960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6f339-e42a-4e74-98ae-07c51cff4431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:11.696239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6f339-e42a-4e74-98ae-07c51cff4431 admin admin] PATH_INFO: `/v3/projects/8739ec17f7e2495c80f8a2cd1aabf41b/users/1bdbd4c9db094556a183be335b764c7b/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:11.718373 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6f339-e42a-4e74-98ae-07c51cff4431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8739ec17f7e2495c80f8a2cd1aabf41b, user_id=1bdbd4c9db094556a183be335b764c7b, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:11.721079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6f339-e42a-4e74-98ae-07c51cff4431 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:11.787334 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1428/2861] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:04:11 2022] PUT /identity/v3/projects/8739ec17f7e2495c80f8a2cd1aabf41b/users/1bdbd4c9db094556a183be335b764c7b/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:04:11.797835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb203b-5883-4a93-9b12-6b5273b45e1f None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:11.798122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb203b-5883-4a93-9b12-6b5273b45e1f None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:11.798397 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb203b-5883-4a93-9b12-6b5273b45e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:11.806491 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:11.825840 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-0beb203b-5883-4a93-9b12-6b5273b45e1f None None] MFA Rules not processed for user `1bdbd4c9db094556a183be335b764c7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:04:11.827044 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-0beb203b-5883-4a93-9b12-6b5273b45e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:11.889925 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1434/2862] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:04:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:04:11.895029 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6f339-e42a-4e74-98ae-07c51cff4431 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:11.957613 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6f339-e42a-4e74-98ae-07c51cff4431 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:11.962465 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477dbf48-c4ce-49a7-9ff9-1e5378a64da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:11.963050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dbf48-c4ce-49a7-9ff9-1e5378a64da0 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:11.963368 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dbf48-c4ce-49a7-9ff9-1e5378a64da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:11.963645 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dbf48-c4ce-49a7-9ff9-1e5378a64da0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:11.964279 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477dbf48-c4ce-49a7-9ff9-1e5378a64da0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:11.966183 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477dbf48-c4ce-49a7-9ff9-1e5378a64da0 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:12.082560 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1429/2863] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:04:11 2022] POST /identity/v3/projects => generated 377 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:04:12.084653 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb203b-5883-4a93-9b12-6b5273b45e1f None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:12.095687 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb203b-5883-4a93-9b12-6b5273b45e1f None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:12.159476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df7b74-574e-423c-83c2-c085946474b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:12.160029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df7b74-574e-423c-83c2-c085946474b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:12.160310 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df7b74-574e-423c-83c2-c085946474b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:12.160584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df7b74-574e-423c-83c2-c085946474b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:12.161323 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df7b74-574e-423c-83c2-c085946474b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:12.163291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df7b74-574e-423c-83c2-c085946474b0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:12.257412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1435/2864] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:04:12 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:04:12.261620 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477dbf48-c4ce-49a7-9ff9-1e5378a64da0 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:12.272504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477dbf48-c4ce-49a7-9ff9-1e5378a64da0 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:12.277353 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb505f-a824-4f02-bc07-c13e4c072b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:12.277957 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb505f-a824-4f02-bc07-c13e4c072b5a admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:12.278239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb505f-a824-4f02-bc07-c13e4c072b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:12.278539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb505f-a824-4f02-bc07-c13e4c072b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:12.279224 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb505f-a824-4f02-bc07-c13e4c072b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:12.280835 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb505f-a824-4f02-bc07-c13e4c072b5a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:12.291962 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1430/2865] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:04:12.296109 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df7b74-574e-423c-83c2-c085946474b0 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:12.306961 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df7b74-574e-423c-83c2-c085946474b0 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:12.311777 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71183f5-9cca-4179-80a3-7510d51967e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:12.316022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71183f5-9cca-4179-80a3-7510d51967e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:12.316022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71183f5-9cca-4179-80a3-7510d51967e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:12.316022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71183f5-9cca-4179-80a3-7510d51967e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:12.316022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71183f5-9cca-4179-80a3-7510d51967e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:12.316022 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71183f5-9cca-4179-80a3-7510d51967e8 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:12.325805 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1436/2866] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:04:12.329730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb505f-a824-4f02-bc07-c13e4c072b5a admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:12.339936 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb505f-a824-4f02-bc07-c13e4c072b5a admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:12.345915 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4afece-1548-4f18-94d3-2e065e59583f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:12.346585 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4afece-1548-4f18-94d3-2e065e59583f admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:12.346866 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4afece-1548-4f18-94d3-2e065e59583f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:12.354417 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4afece-1548-4f18-94d3-2e065e59583f admin admin] PATH_INFO: `/v3/projects/5568517018f64840b2c23a0dc923033b/users/e3b787ec3f584060a4c9543c150c0fb4/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:12.363329 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4afece-1548-4f18-94d3-2e065e59583f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5568517018f64840b2c23a0dc923033b, user_id=e3b787ec3f584060a4c9543c150c0fb4, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:12.365978 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4afece-1548-4f18-94d3-2e065e59583f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:12.432868 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1431/2867] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:04:12 2022] PUT /identity/v3/projects/5568517018f64840b2c23a0dc923033b/users/e3b787ec3f584060a4c9543c150c0fb4/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:04:12.444538 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b7e90-d030-4562-8553-9615d07c065b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:12.444825 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b7e90-d030-4562-8553-9615d07c065b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:12.445102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b7e90-d030-4562-8553-9615d07c065b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:12.473667 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-247b7e90-d030-4562-8553-9615d07c065b None None] MFA Rules not processed for user `e3b787ec3f584060a4c9543c150c0fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:04:12.474711 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-247b7e90-d030-4562-8553-9615d07c065b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:12.541258 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1437/2868] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:04:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:04:12.545960 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4afece-1548-4f18-94d3-2e065e59583f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:12.555838 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:12.609094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4afece-1548-4f18-94d3-2e065e59583f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:12.614522 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e64cf-ae2f-40b6-b8e4-cf2d3ac8a754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:12.615179 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e64cf-ae2f-40b6-b8e4-cf2d3ac8a754 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:12.615469 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e64cf-ae2f-40b6-b8e4-cf2d3ac8a754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:12.615741 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e64cf-ae2f-40b6-b8e4-cf2d3ac8a754 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:12.616371 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e64cf-ae2f-40b6-b8e4-cf2d3ac8a754 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:12.618262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e64cf-ae2f-40b6-b8e4-cf2d3ac8a754 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:12.701922 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1432/2869] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:04:12 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:04:12.706004 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b7e90-d030-4562-8553-9615d07c065b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:12.711465 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:12.717288 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b7e90-d030-4562-8553-9615d07c065b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:12.775029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b04f81-f774-466c-a76d-f0902a95c58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:12.775680 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b04f81-f774-466c-a76d-f0902a95c58e admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:12.776352 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b04f81-f774-466c-a76d-f0902a95c58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:12.776626 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b04f81-f774-466c-a76d-f0902a95c58e admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:12.777263 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b04f81-f774-466c-a76d-f0902a95c58e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:12.779192 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b04f81-f774-466c-a76d-f0902a95c58e admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:12.850038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1438/2870] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:04:12 2022] POST /identity/v3/users => generated 295 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:04:12.854333 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e64cf-ae2f-40b6-b8e4-cf2d3ac8a754 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:12.864980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e64cf-ae2f-40b6-b8e4-cf2d3ac8a754 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:12.869581 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651b275-6f9a-4f3d-b623-09efff4cb218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:12.870176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651b275-6f9a-4f3d-b623-09efff4cb218 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:12.870456 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651b275-6f9a-4f3d-b623-09efff4cb218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:12.870730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651b275-6f9a-4f3d-b623-09efff4cb218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:12.871396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651b275-6f9a-4f3d-b623-09efff4cb218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:12.873064 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651b275-6f9a-4f3d-b623-09efff4cb218 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:12.886834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1433/2871] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:04:12.891438 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b04f81-f774-466c-a76d-f0902a95c58e admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:12.902272 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b04f81-f774-466c-a76d-f0902a95c58e admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:12.906882 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef3caf4-f816-4eb9-8d18-c43c8c3914c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:12.907583 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3caf4-f816-4eb9-8d18-c43c8c3914c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:12.907919 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3caf4-f816-4eb9-8d18-c43c8c3914c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:12.908714 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3caf4-f816-4eb9-8d18-c43c8c3914c7 admin admin] PATH_INFO: `/v3/projects/f5d46ff4f80b4501808c7b4d1df0970f/users/daa06dd87fc545d4a3dcd486d4f7ba9d/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:12.923360 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3caf4-f816-4eb9-8d18-c43c8c3914c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5d46ff4f80b4501808c7b4d1df0970f, user_id=daa06dd87fc545d4a3dcd486d4f7ba9d, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:12.926027 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3caf4-f816-4eb9-8d18-c43c8c3914c7 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:12.977171 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1439/2872] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:04:12 2022] PUT /identity/v3/projects/f5d46ff4f80b4501808c7b4d1df0970f/users/daa06dd87fc545d4a3dcd486d4f7ba9d/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:04:12.981704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651b275-6f9a-4f3d-b623-09efff4cb218 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:12.992449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651b275-6f9a-4f3d-b623-09efff4cb218 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:13.060611 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182be11-5b9b-40c7-8325-9ffd8d79f963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:13.061242 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182be11-5b9b-40c7-8325-9ffd8d79f963 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:13.061526 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182be11-5b9b-40c7-8325-9ffd8d79f963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:13.061802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182be11-5b9b-40c7-8325-9ffd8d79f963 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:13.062467 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182be11-5b9b-40c7-8325-9ffd8d79f963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:13.064274 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182be11-5b9b-40c7-8325-9ffd8d79f963 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:13.075929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1434/2873] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:04:13.081170 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef3caf4-f816-4eb9-8d18-c43c8c3914c7 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:13.091700 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef3caf4-f816-4eb9-8d18-c43c8c3914c7 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:13.097002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb665a-f94a-48cb-83ba-888d6f4dc1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:13.097671 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb665a-f94a-48cb-83ba-888d6f4dc1b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:13.097948 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb665a-f94a-48cb-83ba-888d6f4dc1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:13.098218 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb665a-f94a-48cb-83ba-888d6f4dc1b2 admin admin] PATH_INFO: `/v3/projects/f5d46ff4f80b4501808c7b4d1df0970f/users/daa06dd87fc545d4a3dcd486d4f7ba9d/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:13.100201 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb665a-f94a-48cb-83ba-888d6f4dc1b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5d46ff4f80b4501808c7b4d1df0970f, user_id=daa06dd87fc545d4a3dcd486d4f7ba9d, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:13.102853 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb665a-f94a-48cb-83ba-888d6f4dc1b2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:13.118013 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1440/2874] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:04:13 2022] PUT /identity/v3/projects/f5d46ff4f80b4501808c7b4d1df0970f/users/daa06dd87fc545d4a3dcd486d4f7ba9d/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:04:13.129505 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa26b1e-b991-4314-8416-3c86b719e374 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:13.130066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa26b1e-b991-4314-8416-3c86b719e374 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:13.130171 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa26b1e-b991-4314-8416-3c86b719e374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:13.165138 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-7aa26b1e-b991-4314-8416-3c86b719e374 None None] MFA Rules not processed for user `daa06dd87fc545d4a3dcd486d4f7ba9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:04:13.166360 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7aa26b1e-b991-4314-8416-3c86b719e374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:13.235870 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1435/2875] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:04:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:04:13.247191 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb665a-f94a-48cb-83ba-888d6f4dc1b2 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:13.248906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f3cb665a-f94a-48cb-83ba-888d6f4dc1b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:13.307685 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb665a-f94a-48cb-83ba-888d6f4dc1b2 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:13.311725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960cb9f-fd93-4a08-8d21-0e1af8cc0e6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:13.312368 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960cb9f-fd93-4a08-8d21-0e1af8cc0e6a service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:13.312653 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960cb9f-fd93-4a08-8d21-0e1af8cc0e6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:13.312931 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960cb9f-fd93-4a08-8d21-0e1af8cc0e6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:13.371501 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960cb9f-fd93-4a08-8d21-0e1af8cc0e6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:13.372939 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960cb9f-fd93-4a08-8d21-0e1af8cc0e6a service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:13.382678 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1441/2876] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:04:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:04:15.281787 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa26b1e-b991-4314-8416-3c86b719e374 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:15.283307 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-7aa26b1e-b991-4314-8416-3c86b719e374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:15.290870 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:15.296510 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa26b1e-b991-4314-8416-3c86b719e374 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:15.303609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab6cca-9899-4c8a-8137-d38498ff209d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:15.303609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab6cca-9899-4c8a-8137-d38498ff209d service cinder] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:15.303609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab6cca-9899-4c8a-8137-d38498ff209d service cinder] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:15.303609 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab6cca-9899-4c8a-8137-d38498ff209d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:15.359629 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab6cca-9899-4c8a-8137-d38498ff209d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:15.361054 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab6cca-9899-4c8a-8137-d38498ff209d service cinder] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:15.370016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1436/2877] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:04:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:04:16.638976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960cb9f-fd93-4a08-8d21-0e1af8cc0e6a service cinder] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:16.648536 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960cb9f-fd93-4a08-8d21-0e1af8cc0e6a service cinder] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:16.652290 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d48eb3-2fa1-4e71-8598-ae42851b9956 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:16.652897 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d48eb3-2fa1-4e71-8598-ae42851b9956 service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:16.653180 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d48eb3-2fa1-4e71-8598-ae42851b9956 service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:16.653458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d48eb3-2fa1-4e71-8598-ae42851b9956 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:16.661158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:16.663417 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d48eb3-2fa1-4e71-8598-ae42851b9956 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:16.664897 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d48eb3-2fa1-4e71-8598-ae42851b9956 service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:16.675289 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1442/2878] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:04:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:04:21.206488 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab6cca-9899-4c8a-8137-d38498ff209d service cinder] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:21.270391 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab6cca-9899-4c8a-8137-d38498ff209d service cinder] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:21.275030 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3442c4-d90d-43e1-95dc-719912d3933f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:21.275708 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3442c4-d90d-43e1-95dc-719912d3933f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:21.275971 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3442c4-d90d-43e1-95dc-719912d3933f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:21.276247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3442c4-d90d-43e1-95dc-719912d3933f admin admin] PATH_INFO: `/v3/users/1bdbd4c9db094556a183be335b764c7b` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:21.277390 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3442c4-d90d-43e1-95dc-719912d3933f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bdbd4c9db094556a183be335b764c7b)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:21.279340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3442c4-d90d-43e1-95dc-719912d3933f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:21.469767 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-6b3442c4-d90d-43e1-95dc-719912d3933f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bdbd4c9db094556a183be335b764c7b'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:21.482590 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-6b3442c4-d90d-43e1-95dc-719912d3933f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bdbd4c9db094556a183be335b764c7b'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:21.490050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-6b3442c4-d90d-43e1-95dc-719912d3933f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bdbd4c9db094556a183be335b764c7b'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:21.551562 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-6b3442c4-d90d-43e1-95dc-719912d3933f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bdbd4c9db094556a183be335b764c7b'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:21.552187 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-6b3442c4-d90d-43e1-95dc-719912d3933f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bdbd4c9db094556a183be335b764c7b'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:21.558500 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1437/2879] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:04:21 2022] DELETE /identity/v3/users/1bdbd4c9db094556a183be335b764c7b => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:04:21.562983 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef25a4-9ff1-43c0-b0c6-e917006e0e34 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:21.563494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef25a4-9ff1-43c0-b0c6-e917006e0e34 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:21.563592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef25a4-9ff1-43c0-b0c6-e917006e0e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:21.584385 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d7ef25a4-9ff1-43c0-b0c6-e917006e0e34 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:04:21.585662 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d7ef25a4-9ff1-43c0-b0c6-e917006e0e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:21.657846 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1443/2880] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:04:21.667578 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3442c4-d90d-43e1-95dc-719912d3933f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:21.668984 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-6b3442c4-d90d-43e1-95dc-719912d3933f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:21.690551 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:21.726449 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3442c4-d90d-43e1-95dc-719912d3933f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:21.730128 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5447c31-fb38-4bce-82f9-8992c6d16d1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:21.730811 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5447c31-fb38-4bce-82f9-8992c6d16d1c service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:21.731097 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5447c31-fb38-4bce-82f9-8992c6d16d1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:21.731399 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5447c31-fb38-4bce-82f9-8992c6d16d1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:21.741063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5447c31-fb38-4bce-82f9-8992c6d16d1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:21.742487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5447c31-fb38-4bce-82f9-8992c6d16d1c service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:21.752116 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1438/2881] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:04:22.137111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef25a4-9ff1-43c0-b0c6-e917006e0e34 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:22.138440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d7ef25a4-9ff1-43c0-b0c6-e917006e0e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:22.151598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef25a4-9ff1-43c0-b0c6-e917006e0e34 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:22.156343 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b7fa1-58d6-4ce5-91a0-26957a48743d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:22.157002 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b7fa1-58d6-4ce5-91a0-26957a48743d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:22.157284 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b7fa1-58d6-4ce5-91a0-26957a48743d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:22.157559 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b7fa1-58d6-4ce5-91a0-26957a48743d admin admin] PATH_INFO: `/v3/projects/8739ec17f7e2495c80f8a2cd1aabf41b` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:22.158707 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b7fa1-58d6-4ce5-91a0-26957a48743d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8739ec17f7e2495c80f8a2cd1aabf41b)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:22.160664 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b7fa1-58d6-4ce5-91a0-26957a48743d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:22.267542 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e22b7fa1-58d6-4ce5-91a0-26957a48743d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8739ec17f7e2495c80f8a2cd1aabf41b', 'request_id': 'req-e22b7fa1-58d6-4ce5-91a0-26957a48743d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:22.323563 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e22b7fa1-58d6-4ce5-91a0-26957a48743d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8739ec17f7e2495c80f8a2cd1aabf41b', 'request_id': 'req-e22b7fa1-58d6-4ce5-91a0-26957a48743d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:22.330721 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e22b7fa1-58d6-4ce5-91a0-26957a48743d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8739ec17f7e2495c80f8a2cd1aabf41b', 'request_id': 'req-e22b7fa1-58d6-4ce5-91a0-26957a48743d'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:22.341921 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e22b7fa1-58d6-4ce5-91a0-26957a48743d admin admin] The token cache is being invalidate because project 8739ec17f7e2495c80f8a2cd1aabf41b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:04:22.342212 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e22b7fa1-58d6-4ce5-91a0-26957a48743d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:22.342856 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-e22b7fa1-58d6-4ce5-91a0-26957a48743d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:22.344458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1444/2882] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:04:22 2022] DELETE /identity/v3/projects/8739ec17f7e2495c80f8a2cd1aabf41b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:04:22.349130 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5447c31-fb38-4bce-82f9-8992c6d16d1c service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:22.359123 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5447c31-fb38-4bce-82f9-8992c6d16d1c service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:22.418950 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04db5a8-afc7-4ec5-8751-b430b59e5020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:22.419633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04db5a8-afc7-4ec5-8751-b430b59e5020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:22.419902 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04db5a8-afc7-4ec5-8751-b430b59e5020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:22.420176 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04db5a8-afc7-4ec5-8751-b430b59e5020 admin admin] PATH_INFO: `/v3/users/e3b787ec3f584060a4c9543c150c0fb4` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:22.421323 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04db5a8-afc7-4ec5-8751-b430b59e5020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3b787ec3f584060a4c9543c150c0fb4)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:22.423281 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04db5a8-afc7-4ec5-8751-b430b59e5020 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:22.588751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a04db5a8-afc7-4ec5-8751-b430b59e5020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3b787ec3f584060a4c9543c150c0fb4'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:22.601850 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a04db5a8-afc7-4ec5-8751-b430b59e5020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3b787ec3f584060a4c9543c150c0fb4'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:22.608589 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a04db5a8-afc7-4ec5-8751-b430b59e5020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3b787ec3f584060a4c9543c150c0fb4'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:22.666063 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a04db5a8-afc7-4ec5-8751-b430b59e5020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3b787ec3f584060a4c9543c150c0fb4'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:22.666679 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a04db5a8-afc7-4ec5-8751-b430b59e5020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3b787ec3f584060a4c9543c150c0fb4'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:22.672861 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1439/2883] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:04:22 2022] DELETE /identity/v3/users/e3b787ec3f584060a4c9543c150c0fb4 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:04:23.119520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b7fa1-58d6-4ce5-91a0-26957a48743d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:23.121101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e22b7fa1-58d6-4ce5-91a0-26957a48743d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:23.137835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b7fa1-58d6-4ce5-91a0-26957a48743d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:23.198109 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70b048-85e5-4812-9595-0f9f5e1b4405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:23.203352 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70b048-85e5-4812-9595-0f9f5e1b4405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:23.203352 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70b048-85e5-4812-9595-0f9f5e1b4405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:23.203352 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70b048-85e5-4812-9595-0f9f5e1b4405 admin admin] PATH_INFO: `/v3/projects/5568517018f64840b2c23a0dc923033b` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:23.203352 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70b048-85e5-4812-9595-0f9f5e1b4405 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5568517018f64840b2c23a0dc923033b)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:23.203352 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70b048-85e5-4812-9595-0f9f5e1b4405 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:23.231197 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:23.342211 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9c70b048-85e5-4812-9595-0f9f5e1b4405 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5568517018f64840b2c23a0dc923033b', 'request_id': 'req-9c70b048-85e5-4812-9595-0f9f5e1b4405'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:23.409071 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9c70b048-85e5-4812-9595-0f9f5e1b4405 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5568517018f64840b2c23a0dc923033b', 'request_id': 'req-9c70b048-85e5-4812-9595-0f9f5e1b4405'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:23.416340 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9c70b048-85e5-4812-9595-0f9f5e1b4405 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5568517018f64840b2c23a0dc923033b', 'request_id': 'req-9c70b048-85e5-4812-9595-0f9f5e1b4405'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:23.427527 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9c70b048-85e5-4812-9595-0f9f5e1b4405 admin admin] The token cache is being invalidate because project 5568517018f64840b2c23a0dc923033b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:04:23.427822 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9c70b048-85e5-4812-9595-0f9f5e1b4405 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:23.428479 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-9c70b048-85e5-4812-9595-0f9f5e1b4405 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:23.429964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1445/2884] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:04:23 2022] DELETE /identity/v3/projects/5568517018f64840b2c23a0dc923033b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:04:23.434151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04db5a8-afc7-4ec5-8751-b430b59e5020 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:23.435591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a04db5a8-afc7-4ec5-8751-b430b59e5020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:23.451030 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04db5a8-afc7-4ec5-8751-b430b59e5020 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:23.491634 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:23.508364 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1c1da-dad9-4533-b80d-c573583af87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:23.509002 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1c1da-dad9-4533-b80d-c573583af87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:23.509283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1c1da-dad9-4533-b80d-c573583af87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:23.509554 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1c1da-dad9-4533-b80d-c573583af87e admin admin] PATH_INFO: `/v3/users/daa06dd87fc545d4a3dcd486d4f7ba9d` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:23.510668 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1c1da-dad9-4533-b80d-c573583af87e admin admin] RBAC: Authorizing `identity:delete_user(user_id=daa06dd87fc545d4a3dcd486d4f7ba9d)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:23.512596 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1c1da-dad9-4533-b80d-c573583af87e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:23.714195 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a4c1c1da-dad9-4533-b80d-c573583af87e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daa06dd87fc545d4a3dcd486d4f7ba9d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:23.727126 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a4c1c1da-dad9-4533-b80d-c573583af87e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daa06dd87fc545d4a3dcd486d4f7ba9d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:23.734094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a4c1c1da-dad9-4533-b80d-c573583af87e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daa06dd87fc545d4a3dcd486d4f7ba9d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:23.912661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a4c1c1da-dad9-4533-b80d-c573583af87e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daa06dd87fc545d4a3dcd486d4f7ba9d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:23.913318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-a4c1c1da-dad9-4533-b80d-c573583af87e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daa06dd87fc545d4a3dcd486d4f7ba9d'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:23.919788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1440/2885] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:04:23 2022] DELETE /identity/v3/users/daa06dd87fc545d4a3dcd486d4f7ba9d => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:04:24.302847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70b048-85e5-4812-9595-0f9f5e1b4405 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:24.304449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9c70b048-85e5-4812-9595-0f9f5e1b4405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:24.320208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70b048-85e5-4812-9595-0f9f5e1b4405 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:24.382206 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbbd4ac-5b2b-4cfa-a641-a58175f70559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:24.382875 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbd4ac-5b2b-4cfa-a641-a58175f70559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:24.383164 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbd4ac-5b2b-4cfa-a641-a58175f70559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:24.383470 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbd4ac-5b2b-4cfa-a641-a58175f70559 admin admin] PATH_INFO: `/v3/projects/f5d46ff4f80b4501808c7b4d1df0970f` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:24.384622 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbbd4ac-5b2b-4cfa-a641-a58175f70559 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5d46ff4f80b4501808c7b4d1df0970f)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:24.386564 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbbd4ac-5b2b-4cfa-a641-a58175f70559 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:24.490740 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3fbbd4ac-5b2b-4cfa-a641-a58175f70559 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5d46ff4f80b4501808c7b4d1df0970f', 'request_id': 'req-3fbbd4ac-5b2b-4cfa-a641-a58175f70559'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:24.552405 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3fbbd4ac-5b2b-4cfa-a641-a58175f70559 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5d46ff4f80b4501808c7b4d1df0970f', 'request_id': 'req-3fbbd4ac-5b2b-4cfa-a641-a58175f70559'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:24.559262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3fbbd4ac-5b2b-4cfa-a641-a58175f70559 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5d46ff4f80b4501808c7b4d1df0970f', 'request_id': 'req-3fbbd4ac-5b2b-4cfa-a641-a58175f70559'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:24.569892 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3fbbd4ac-5b2b-4cfa-a641-a58175f70559 admin admin] The token cache is being invalidate because project f5d46ff4f80b4501808c7b4d1df0970f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:04:24.570186 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3fbbd4ac-5b2b-4cfa-a641-a58175f70559 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:24.570841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-3fbbd4ac-5b2b-4cfa-a641-a58175f70559 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:04:24.572341 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1446/2886] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:04:24 2022] DELETE /identity/v3/projects/f5d46ff4f80b4501808c7b4d1df0970f => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:04:24.577681 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a541569-2f09-4934-8761-02e8cf21ea04 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:24.578209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a541569-2f09-4934-8761-02e8cf21ea04 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:24.578209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a541569-2f09-4934-8761-02e8cf21ea04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:24.599742 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-2a541569-2f09-4934-8761-02e8cf21ea04 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:04:24.600968 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2a541569-2f09-4934-8761-02e8cf21ea04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:24.660460 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1441/2887] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:04:24.668958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4556b-d943-4711-aec6-4fea0efa86d7 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:24.669249 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4556b-d943-4711-aec6-4fea0efa86d7 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:24.669530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4556b-d943-4711-aec6-4fea0efa86d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:24.691832 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-68c4556b-d943-4711-aec6-4fea0efa86d7 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:04:24.693101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-68c4556b-d943-4711-aec6-4fea0efa86d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:24.701684 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1447/2888] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:04:24.705996 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a541569-2f09-4934-8761-02e8cf21ea04 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:24.716422 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a541569-2f09-4934-8761-02e8cf21ea04 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:24.721050 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd228d-4e95-47d0-b927-689b50093497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:24.721645 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd228d-4e95-47d0-b927-689b50093497 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:24.721924 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd228d-4e95-47d0-b927-689b50093497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:24.722198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd228d-4e95-47d0-b927-689b50093497 admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:24.722842 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd228d-4e95-47d0-b927-689b50093497 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:04:24.723163 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd228d-4e95-47d0-b927-689b50093497 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:24.725354 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd228d-4e95-47d0-b927-689b50093497 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:24.737459 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1442/2889] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:04:24.742098 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4556b-d943-4711-aec6-4fea0efa86d7 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:24.751328 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4556b-d943-4711-aec6-4fea0efa86d7 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:24.756049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcf721-541e-4891-b606-08dccd1129a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:24.756642 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcf721-541e-4891-b606-08dccd1129a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:24.756917 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcf721-541e-4891-b606-08dccd1129a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:24.757186 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcf721-541e-4891-b606-08dccd1129a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:24.757812 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbcf721-541e-4891-b606-08dccd1129a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:24.759704 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbcf721-541e-4891-b606-08dccd1129a3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:24.908309 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1448/2890] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:04:24 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:04:24.912841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd228d-4e95-47d0-b927-689b50093497 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:24.922902 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd228d-4e95-47d0-b927-689b50093497 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:24.966903 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:24.978982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82d7b4-ae83-4f0e-937b-c8c003d4b15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:24.979610 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82d7b4-ae83-4f0e-937b-c8c003d4b15d admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:24.979888 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82d7b4-ae83-4f0e-937b-c8c003d4b15d admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:24.980155 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82d7b4-ae83-4f0e-937b-c8c003d4b15d admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:24.980781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82d7b4-ae83-4f0e-937b-c8c003d4b15d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:24.982806 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82d7b4-ae83-4f0e-937b-c8c003d4b15d admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:25.060633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1443/2891] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:04:24 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:04:25.065040 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcf721-541e-4891-b606-08dccd1129a3 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:25.076463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcf721-541e-4891-b606-08dccd1129a3 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:25.081271 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790c80a-e1c4-42f3-a047-3d6da4b75c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:25.081902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790c80a-e1c4-42f3-a047-3d6da4b75c3c admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:25.082197 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790c80a-e1c4-42f3-a047-3d6da4b75c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:25.082449 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790c80a-e1c4-42f3-a047-3d6da4b75c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:25.083745 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790c80a-e1c4-42f3-a047-3d6da4b75c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:25.085400 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790c80a-e1c4-42f3-a047-3d6da4b75c3c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:25.091273 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:25.096229 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1449/2892] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:04:25.105567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82d7b4-ae83-4f0e-937b-c8c003d4b15d admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:25.116196 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82d7b4-ae83-4f0e-937b-c8c003d4b15d admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:25.120980 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d17798-0924-476d-a074-0018cb6ae96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:25.125642 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d17798-0924-476d-a074-0018cb6ae96d admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:25.125642 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d17798-0924-476d-a074-0018cb6ae96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:25.125642 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d17798-0924-476d-a074-0018cb6ae96d admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:25.125642 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d17798-0924-476d-a074-0018cb6ae96d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:25.125642 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d17798-0924-476d-a074-0018cb6ae96d admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:25.136282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1444/2893] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:04:25.140412 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790c80a-e1c4-42f3-a047-3d6da4b75c3c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:25.151435 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790c80a-e1c4-42f3-a047-3d6da4b75c3c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:25.156155 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8448dd-e6fc-4dcd-abb1-c389f01bc423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:25.156813 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8448dd-e6fc-4dcd-abb1-c389f01bc423 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:25.157303 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8448dd-e6fc-4dcd-abb1-c389f01bc423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:25.157808 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8448dd-e6fc-4dcd-abb1-c389f01bc423 admin admin] PATH_INFO: `/v3/projects/b7d811c8ed584ddb92b4642d85e8db00/users/ab1e5eb74e784f53a9250b9a7a2220c1/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:25.172806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8448dd-e6fc-4dcd-abb1-c389f01bc423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d811c8ed584ddb92b4642d85e8db00, user_id=ab1e5eb74e784f53a9250b9a7a2220c1, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:25.175442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8448dd-e6fc-4dcd-abb1-c389f01bc423 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:25.237490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1450/2894] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:04:25 2022] PUT /identity/v3/projects/b7d811c8ed584ddb92b4642d85e8db00/users/ab1e5eb74e784f53a9250b9a7a2220c1/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:04:25.242848 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd540d-a94a-4ab2-8080-35dfdd6316ea None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:25.243140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd540d-a94a-4ab2-8080-35dfdd6316ea None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:25.243438 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd540d-a94a-4ab2-8080-35dfdd6316ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:25.262821 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-d9bd540d-a94a-4ab2-8080-35dfdd6316ea None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:04:25.264080 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d9bd540d-a94a-4ab2-8080-35dfdd6316ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:25.322975 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1445/2895] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:04:25.332598 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8448dd-e6fc-4dcd-abb1-c389f01bc423 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:25.334947 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ec8448dd-e6fc-4dcd-abb1-c389f01bc423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:25.397456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8448dd-e6fc-4dcd-abb1-c389f01bc423 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:25.401106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da295c07-838b-4645-b895-91e437a5aebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:25.401720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da295c07-838b-4645-b895-91e437a5aebc service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:25.401999 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da295c07-838b-4645-b895-91e437a5aebc service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:25.402270 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da295c07-838b-4645-b895-91e437a5aebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:25.411968 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da295c07-838b-4645-b895-91e437a5aebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:25.413403 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da295c07-838b-4645-b895-91e437a5aebc service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:25.423202 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1451/2896] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:04:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:04:30.744000 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756df63-1e5d-4112-a5cd-995113127f40 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:30.744518 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756df63-1e5d-4112-a5cd-995113127f40 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:30.744628 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756df63-1e5d-4112-a5cd-995113127f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:30.774751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-2756df63-1e5d-4112-a5cd-995113127f40 None None] MFA Rules not processed for user `ab1e5eb74e784f53a9250b9a7a2220c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:04:30.776022 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-2756df63-1e5d-4112-a5cd-995113127f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:30.844474 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1446/2897] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:04:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:04:30.849080 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da295c07-838b-4645-b895-91e437a5aebc service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:30.859185 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da295c07-838b-4645-b895-91e437a5aebc service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:30.864725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2545864-792b-424a-9f03-cbb49ce0cfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:30.865548 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2545864-792b-424a-9f03-cbb49ce0cfea admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:30.865741 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2545864-792b-424a-9f03-cbb49ce0cfea admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:30.866011 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2545864-792b-424a-9f03-cbb49ce0cfea admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:30.866651 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2545864-792b-424a-9f03-cbb49ce0cfea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:30.868678 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2545864-792b-424a-9f03-cbb49ce0cfea admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:30.989136 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1452/2898] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:04:30 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:04:30.993354 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756df63-1e5d-4112-a5cd-995113127f40 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:31.005368 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756df63-1e5d-4112-a5cd-995113127f40 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:31.061481 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:31.068651 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90893019-c7aa-4c95-8c08-1fe2db32e969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:31.069140 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90893019-c7aa-4c95-8c08-1fe2db32e969 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:31.069426 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90893019-c7aa-4c95-8c08-1fe2db32e969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:31.069696 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90893019-c7aa-4c95-8c08-1fe2db32e969 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:31.070343 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90893019-c7aa-4c95-8c08-1fe2db32e969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:31.072494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90893019-c7aa-4c95-8c08-1fe2db32e969 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:31.158897 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1447/2899] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:04:30 2022] POST /identity/v3/users => generated 295 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:04:31.163442 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2545864-792b-424a-9f03-cbb49ce0cfea admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:31.174534 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2545864-792b-424a-9f03-cbb49ce0cfea admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:31.180147 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa04163-78e1-41a4-8799-f7492c5c1c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:31.180756 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa04163-78e1-41a4-8799-f7492c5c1c73 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:31.181112 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa04163-78e1-41a4-8799-f7492c5c1c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:31.181380 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa04163-78e1-41a4-8799-f7492c5c1c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:31.182021 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa04163-78e1-41a4-8799-f7492c5c1c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:31.183647 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa04163-78e1-41a4-8799-f7492c5c1c73 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:31.195093 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1453/2900] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:04:31.199915 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90893019-c7aa-4c95-8c08-1fe2db32e969 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:31.210227 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90893019-c7aa-4c95-8c08-1fe2db32e969 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:31.215015 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45984332-3c0b-4630-8d7f-777d1b916bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:31.215749 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45984332-3c0b-4630-8d7f-777d1b916bac admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:31.216022 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45984332-3c0b-4630-8d7f-777d1b916bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:31.216321 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45984332-3c0b-4630-8d7f-777d1b916bac admin admin] PATH_INFO: `/v3/projects/f7bb01d8ee844b70969677665fb154c8/users/44eab873d87949369e77d39ff32507fa/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:31.232409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45984332-3c0b-4630-8d7f-777d1b916bac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7bb01d8ee844b70969677665fb154c8, user_id=44eab873d87949369e77d39ff32507fa, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:31.235136 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45984332-3c0b-4630-8d7f-777d1b916bac admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:31.283633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1448/2901] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:04:31 2022] PUT /identity/v3/projects/f7bb01d8ee844b70969677665fb154c8/users/44eab873d87949369e77d39ff32507fa/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:04:31.288203 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa04163-78e1-41a4-8799-f7492c5c1c73 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:31.298269 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa04163-78e1-41a4-8799-f7492c5c1c73 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:31.358511 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeff214-1463-4a18-ad67-d3b67b227079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:31.359158 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeff214-1463-4a18-ad67-d3b67b227079 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:31.359702 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeff214-1463-4a18-ad67-d3b67b227079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:31.360058 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeff214-1463-4a18-ad67-d3b67b227079 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:31.360618 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeff214-1463-4a18-ad67-d3b67b227079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:31.362271 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeff214-1463-4a18-ad67-d3b67b227079 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:31.373409 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1454/2902] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:04:31.377593 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45984332-3c0b-4630-8d7f-777d1b916bac admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:31.388084 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45984332-3c0b-4630-8d7f-777d1b916bac admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:31.393152 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9547941-106c-45c7-8397-a5c826890a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:31.393819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9547941-106c-45c7-8397-a5c826890a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:31.394102 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9547941-106c-45c7-8397-a5c826890a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:31.394383 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9547941-106c-45c7-8397-a5c826890a72 admin admin] PATH_INFO: `/v3/projects/f7bb01d8ee844b70969677665fb154c8/users/44eab873d87949369e77d39ff32507fa/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:31.396381 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9547941-106c-45c7-8397-a5c826890a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7bb01d8ee844b70969677665fb154c8, user_id=44eab873d87949369e77d39ff32507fa, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:31.399208 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9547941-106c-45c7-8397-a5c826890a72 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:31.411883 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1449/2903] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:04:31 2022] PUT /identity/v3/projects/f7bb01d8ee844b70969677665fb154c8/users/44eab873d87949369e77d39ff32507fa/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:04:36.949372 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a937d9d-6dc8-4f76-b20d-8621aeddb17a None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:36.949834 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a937d9d-6dc8-4f76-b20d-8621aeddb17a None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:36.949951 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a937d9d-6dc8-4f76-b20d-8621aeddb17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:36.987737 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-9a937d9d-6dc8-4f76-b20d-8621aeddb17a None None] MFA Rules not processed for user `44eab873d87949369e77d39ff32507fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:04:36.988983 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9a937d9d-6dc8-4f76-b20d-8621aeddb17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:37.021596 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:37.064413 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1455/2904] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:04:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:04:37.083645 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9547941-106c-45c7-8397-a5c826890a72 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:04:37.085657 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d9547941-106c-45c7-8397-a5c826890a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:37.169315 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9547941-106c-45c7-8397-a5c826890a72 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:04:37.175654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804d21f-248f-4a89-b71b-844fda771c49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:04:37.176278 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804d21f-248f-4a89-b71b-844fda771c49 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:37.176550 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804d21f-248f-4a89-b71b-844fda771c49 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:37.176824 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804d21f-248f-4a89-b71b-844fda771c49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:37.237819 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804d21f-248f-4a89-b71b-844fda771c49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:04:37.239511 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804d21f-248f-4a89-b71b-844fda771c49 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:04:37.250352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1450/2905] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:04:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:04:47.101228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=15 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:47.244921 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=74,keystone:INSERT=5 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:04:55.688403 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1f39e-cfb7-4457-8853-0a9dc196f410 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:04:55.688992 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1f39e-cfb7-4457-8853-0a9dc196f410 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:04:55.689152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1f39e-cfb7-4457-8853-0a9dc196f410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:04:55.709089 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-76f1f39e-cfb7-4457-8853-0a9dc196f410 None None] MFA Rules not processed for user `ab1e5eb74e784f53a9250b9a7a2220c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:04:55.710317 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-76f1f39e-cfb7-4457-8853-0a9dc196f410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:04:55.717710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1456/2906] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:04:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 12:05:00.524381 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804d21f-248f-4a89-b71b-844fda771c49 service nova] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:00.528434 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-4804d21f-248f-4a89-b71b-844fda771c49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:00.593079 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804d21f-248f-4a89-b71b-844fda771c49 service nova] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:00.596781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a2c7a-3b51-47ba-9433-4850a43729d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:00.597382 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a2c7a-3b51-47ba-9433-4850a43729d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:00.597654 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a2c7a-3b51-47ba-9433-4850a43729d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:00.597953 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a2c7a-3b51-47ba-9433-4850a43729d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:00.600329 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-227a2c7a-3b51-47ba-9433-4850a43729d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:00.669405 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a2c7a-3b51-47ba-9433-4850a43729d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:00.670947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a2c7a-3b51-47ba-9433-4850a43729d3 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:00.682816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1451/2907] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:05:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:05:04.674133 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1f39e-cfb7-4457-8853-0a9dc196f410 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:04.675659 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-76f1f39e-cfb7-4457-8853-0a9dc196f410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:04.689716 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1f39e-cfb7-4457-8853-0a9dc196f410 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:04.694298 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce9718-813c-4015-99d0-9d2f82a71eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:04.694924 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce9718-813c-4015-99d0-9d2f82a71eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:04.695226 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce9718-813c-4015-99d0-9d2f82a71eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:04.695500 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce9718-813c-4015-99d0-9d2f82a71eb2 admin admin] PATH_INFO: `/v3/users/7483b0e52765411d8444dfdd46549c5f` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:04.696608 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce9718-813c-4015-99d0-9d2f82a71eb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7483b0e52765411d8444dfdd46549c5f)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:04.698541 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce9718-813c-4015-99d0-9d2f82a71eb2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:04.839774 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-dcce9718-813c-4015-99d0-9d2f82a71eb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7483b0e52765411d8444dfdd46549c5f'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:04.852362 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-dcce9718-813c-4015-99d0-9d2f82a71eb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7483b0e52765411d8444dfdd46549c5f'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:04.859475 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-dcce9718-813c-4015-99d0-9d2f82a71eb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7483b0e52765411d8444dfdd46549c5f'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:04.928095 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-dcce9718-813c-4015-99d0-9d2f82a71eb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7483b0e52765411d8444dfdd46549c5f'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:04.928762 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-dcce9718-813c-4015-99d0-9d2f82a71eb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7483b0e52765411d8444dfdd46549c5f'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:04.935746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1457/2908] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:05:04 2022] DELETE /identity/v3/users/7483b0e52765411d8444dfdd46549c5f => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:05.175810 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a2c7a-3b51-47ba-9433-4850a43729d3 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:05.189342 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a2c7a-3b51-47ba-9433-4850a43729d3 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:05.244349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e391d-8266-44a8-aa8d-a556680b54d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:05.244982 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e391d-8266-44a8-aa8d-a556680b54d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:05.245267 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e391d-8266-44a8-aa8d-a556680b54d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:05.245546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e391d-8266-44a8-aa8d-a556680b54d3 admin admin] PATH_INFO: `/v3/projects/edc640eda36b41d8b9cc59c635992d5b` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:05.246678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e391d-8266-44a8-aa8d-a556680b54d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edc640eda36b41d8b9cc59c635992d5b)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:05.248682 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e391d-8266-44a8-aa8d-a556680b54d3 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:05.375811 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-440e391d-8266-44a8-aa8d-a556680b54d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edc640eda36b41d8b9cc59c635992d5b', 'request_id': 'req-440e391d-8266-44a8-aa8d-a556680b54d3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:05.378133 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:05.383097 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-440e391d-8266-44a8-aa8d-a556680b54d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edc640eda36b41d8b9cc59c635992d5b', 'request_id': 'req-440e391d-8266-44a8-aa8d-a556680b54d3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:05.471342 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-440e391d-8266-44a8-aa8d-a556680b54d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edc640eda36b41d8b9cc59c635992d5b', 'request_id': 'req-440e391d-8266-44a8-aa8d-a556680b54d3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:05.483497 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-440e391d-8266-44a8-aa8d-a556680b54d3 admin admin] The token cache is being invalidate because project edc640eda36b41d8b9cc59c635992d5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:05:05.483788 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-440e391d-8266-44a8-aa8d-a556680b54d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:05.486647 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-440e391d-8266-44a8-aa8d-a556680b54d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:05.486647 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1452/2909] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:05:05 2022] DELETE /identity/v3/projects/edc640eda36b41d8b9cc59c635992d5b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:05.491486 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417b09a-3369-4485-adbf-123197723555 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:05.491772 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417b09a-3369-4485-adbf-123197723555 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:05.492045 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417b09a-3369-4485-adbf-123197723555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:05.512091 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-6417b09a-3369-4485-adbf-123197723555 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:05.513279 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-6417b09a-3369-4485-adbf-123197723555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:05.572087 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1458/2910] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:05.580234 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c9554-295e-4f98-a696-d76e65ed4356 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:05.580523 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c9554-295e-4f98-a696-d76e65ed4356 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:05.580801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c9554-295e-4f98-a696-d76e65ed4356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:05.601461 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-d47c9554-295e-4f98-a696-d76e65ed4356 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:05.602681 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d47c9554-295e-4f98-a696-d76e65ed4356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:05.610754 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1453/2911] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:05.615673 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417b09a-3369-4485-adbf-123197723555 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:05.625533 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417b09a-3369-4485-adbf-123197723555 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:05.630111 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d69c04-72e0-48f4-b662-50ba1548957d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:05.630710 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d69c04-72e0-48f4-b662-50ba1548957d admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:05.630995 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d69c04-72e0-48f4-b662-50ba1548957d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:05.631301 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d69c04-72e0-48f4-b662-50ba1548957d admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:05.631946 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d69c04-72e0-48f4-b662-50ba1548957d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:05:05.632252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d69c04-72e0-48f4-b662-50ba1548957d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:05.633849 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d69c04-72e0-48f4-b662-50ba1548957d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:05.645807 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1459/2912] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:05:05.650031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c9554-295e-4f98-a696-d76e65ed4356 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:05.659431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c9554-295e-4f98-a696-d76e65ed4356 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:05.664658 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103180f9-30d6-4812-8bfe-49c988591fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:05.665259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103180f9-30d6-4812-8bfe-49c988591fff admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:05.665534 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103180f9-30d6-4812-8bfe-49c988591fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:05.665802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103180f9-30d6-4812-8bfe-49c988591fff admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:05.666442 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103180f9-30d6-4812-8bfe-49c988591fff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:05.668374 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103180f9-30d6-4812-8bfe-49c988591fff admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:05.752656 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1454/2913] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:05:05 2022] POST /identity/v3/projects => generated 401 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:05:05.756835 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d69c04-72e0-48f4-b662-50ba1548957d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:05.767527 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d69c04-72e0-48f4-b662-50ba1548957d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:05.818061 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:05.827106 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cea92-856e-40d9-90b9-028b6e105696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:05.828941 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cea92-856e-40d9-90b9-028b6e105696 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:05.828941 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cea92-856e-40d9-90b9-028b6e105696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:05.828941 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cea92-856e-40d9-90b9-028b6e105696 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:05.828941 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838cea92-856e-40d9-90b9-028b6e105696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:05.830967 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838cea92-856e-40d9-90b9-028b6e105696 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:05.929321 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1460/2914] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:05:05 2022] POST /identity/v3/users => generated 301 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:05:05.934383 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103180f9-30d6-4812-8bfe-49c988591fff admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:05.953075 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103180f9-30d6-4812-8bfe-49c988591fff admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:05.961499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4482f7b-36b6-4f7a-802e-d0717a4e911e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:05.962319 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4482f7b-36b6-4f7a-802e-d0717a4e911e admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:05.962715 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4482f7b-36b6-4f7a-802e-d0717a4e911e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:05.963282 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4482f7b-36b6-4f7a-802e-d0717a4e911e admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:05.963938 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4482f7b-36b6-4f7a-802e-d0717a4e911e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:05.965539 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4482f7b-36b6-4f7a-802e-d0717a4e911e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:05.976627 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1455/2915] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:05:05.982319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cea92-856e-40d9-90b9-028b6e105696 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:05.992854 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cea92-856e-40d9-90b9-028b6e105696 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:05.997647 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f2f7d-919c-491d-9f48-d73b47dc680c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:05.998291 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f2f7d-919c-491d-9f48-d73b47dc680c admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:05.998650 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f2f7d-919c-491d-9f48-d73b47dc680c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:05.998935 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f2f7d-919c-491d-9f48-d73b47dc680c admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:05.999614 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f2f7d-919c-491d-9f48-d73b47dc680c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:06.001225 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f2f7d-919c-491d-9f48-d73b47dc680c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:06.012023 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1461/2916] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:05:06.016683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4482f7b-36b6-4f7a-802e-d0717a4e911e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:06.027155 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4482f7b-36b6-4f7a-802e-d0717a4e911e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:06.031863 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddc313-50b2-4bfd-a256-a33b626eb66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:06.032507 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddc313-50b2-4bfd-a256-a33b626eb66e admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:06.032789 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddc313-50b2-4bfd-a256-a33b626eb66e admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:06.033065 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddc313-50b2-4bfd-a256-a33b626eb66e admin admin] PATH_INFO: `/v3/projects/f229b5419d4c4a53a5e9689e96b0f87d/users/7460021c09bc4d23970b5214d84f0c92/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:06.048505 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddc313-50b2-4bfd-a256-a33b626eb66e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f229b5419d4c4a53a5e9689e96b0f87d, user_id=7460021c09bc4d23970b5214d84f0c92, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:06.051404 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddc313-50b2-4bfd-a256-a33b626eb66e admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:06.123494 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1456/2917] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:05:06 2022] PUT /identity/v3/projects/f229b5419d4c4a53a5e9689e96b0f87d/users/7460021c09bc4d23970b5214d84f0c92/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:06.134782 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52d4c0-e20a-4d61-b975-98fe00a0e0d2 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:06.135164 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52d4c0-e20a-4d61-b975-98fe00a0e0d2 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:06.135530 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52d4c0-e20a-4d61-b975-98fe00a0e0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:06.168922 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-ac52d4c0-e20a-4d61-b975-98fe00a0e0d2 None None] MFA Rules not processed for user `7460021c09bc4d23970b5214d84f0c92`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:06.170166 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-ac52d4c0-e20a-4d61-b975-98fe00a0e0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:06.241015 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1462/2918] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:05:06 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:06.245622 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddc313-50b2-4bfd-a256-a33b626eb66e admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:06.309073 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddc313-50b2-4bfd-a256-a33b626eb66e admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:06.313722 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c6da0-3c36-4269-90f3-b84533b4aeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:06.314326 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c6da0-3c36-4269-90f3-b84533b4aeb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:06.314613 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c6da0-3c36-4269-90f3-b84533b4aeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:06.314894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c6da0-3c36-4269-90f3-b84533b4aeb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:06.315567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c6da0-3c36-4269-90f3-b84533b4aeb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:06.317986 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c6da0-3c36-4269-90f3-b84533b4aeb4 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:06.387184 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1457/2919] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:05:06 2022] POST /identity/v3/projects => generated 405 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:05:06.391730 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52d4c0-e20a-4d61-b975-98fe00a0e0d2 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:06.403550 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52d4c0-e20a-4d61-b975-98fe00a0e0d2 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:06.464164 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ce4da-e8a5-4a71-8ac4-862eb785b33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:06.464775 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ce4da-e8a5-4a71-8ac4-862eb785b33d admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:06.465079 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ce4da-e8a5-4a71-8ac4-862eb785b33d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:06.465358 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ce4da-e8a5-4a71-8ac4-862eb785b33d admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:06.465980 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ce4da-e8a5-4a71-8ac4-862eb785b33d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:06.467954 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ce4da-e8a5-4a71-8ac4-862eb785b33d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:06.534212 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1463/2920] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:05:06 2022] POST /identity/v3/users => generated 309 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:05:06.538197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c6da0-3c36-4269-90f3-b84533b4aeb4 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:06.548764 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c6da0-3c36-4269-90f3-b84533b4aeb4 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:06.553666 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198ca81-efa3-47b9-95b0-37d83b28005b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:06.554273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198ca81-efa3-47b9-95b0-37d83b28005b admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:06.554553 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198ca81-efa3-47b9-95b0-37d83b28005b admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:06.554826 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198ca81-efa3-47b9-95b0-37d83b28005b admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:06.555496 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198ca81-efa3-47b9-95b0-37d83b28005b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:06.557203 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198ca81-efa3-47b9-95b0-37d83b28005b admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:06.568781 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1458/2921] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:05:06.573142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ce4da-e8a5-4a71-8ac4-862eb785b33d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:06.584068 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ce4da-e8a5-4a71-8ac4-862eb785b33d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:06.588927 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a7782-c453-4de4-adde-ec68e01a7dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:06.589783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a7782-c453-4de4-adde-ec68e01a7dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:06.590064 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a7782-c453-4de4-adde-ec68e01a7dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:06.590340 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a7782-c453-4de4-adde-ec68e01a7dc2 admin admin] PATH_INFO: `/v3/projects/1fb11f72be4d4d74840075ee2cea0587/users/dcef8e47dda14870b99503e41aa54e2f/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:06.606460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a7782-c453-4de4-adde-ec68e01a7dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb11f72be4d4d74840075ee2cea0587, user_id=dcef8e47dda14870b99503e41aa54e2f, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:06.609252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a7782-c453-4de4-adde-ec68e01a7dc2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:06.675849 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1464/2922] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:05:06 2022] PUT /identity/v3/projects/1fb11f72be4d4d74840075ee2cea0587/users/dcef8e47dda14870b99503e41aa54e2f/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:06.680592 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198ca81-efa3-47b9-95b0-37d83b28005b admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:06.691031 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198ca81-efa3-47b9-95b0-37d83b28005b admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:06.741307 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:06.751678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476e882-92ca-46fc-afc6-c8d043292b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:06.752298 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476e882-92ca-46fc-afc6-c8d043292b26 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:06.752579 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476e882-92ca-46fc-afc6-c8d043292b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:06.752854 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476e882-92ca-46fc-afc6-c8d043292b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:06.753571 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476e882-92ca-46fc-afc6-c8d043292b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:06.755894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476e882-92ca-46fc-afc6-c8d043292b26 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:06.766151 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1459/2923] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:05:06.770391 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a7782-c453-4de4-adde-ec68e01a7dc2 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:06.781273 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a7782-c453-4de4-adde-ec68e01a7dc2 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:06.786007 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fd585-d68f-4a3f-8d56-f7458bbcbe29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:06.786665 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fd585-d68f-4a3f-8d56-f7458bbcbe29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:06.786950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fd585-d68f-4a3f-8d56-f7458bbcbe29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:06.787251 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fd585-d68f-4a3f-8d56-f7458bbcbe29 admin admin] PATH_INFO: `/v3/projects/1fb11f72be4d4d74840075ee2cea0587/users/dcef8e47dda14870b99503e41aa54e2f/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:06.789153 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fd585-d68f-4a3f-8d56-f7458bbcbe29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb11f72be4d4d74840075ee2cea0587, user_id=dcef8e47dda14870b99503e41aa54e2f, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:06.791862 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fd585-d68f-4a3f-8d56-f7458bbcbe29 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:06.802821 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1465/2924] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:05:06 2022] PUT /identity/v3/projects/1fb11f72be4d4d74840075ee2cea0587/users/dcef8e47dda14870b99503e41aa54e2f/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:06.816191 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8255842-571a-49d1-8afa-c9f55704365b None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:06.816191 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8255842-571a-49d1-8afa-c9f55704365b None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:06.816191 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8255842-571a-49d1-8afa-c9f55704365b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:06.843275 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-a8255842-571a-49d1-8afa-c9f55704365b None None] MFA Rules not processed for user `dcef8e47dda14870b99503e41aa54e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:06.844506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-a8255842-571a-49d1-8afa-c9f55704365b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:06.914874 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1460/2925] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:05:06 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:06.925478 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fd585-d68f-4a3f-8d56-f7458bbcbe29 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:06.927828 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-091fd585-d68f-4a3f-8d56-f7458bbcbe29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:06.943336 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:06.985845 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fd585-d68f-4a3f-8d56-f7458bbcbe29 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:06.990196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d8104-741d-4efb-bed4-88a738648c2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ab94f6e08644fa7a76b9057e46d122d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:06.990796 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d8104-741d-4efb-bed4-88a738648c2c service cinder] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:06.991077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d8104-741d-4efb-bed4-88a738648c2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:06.991380 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d8104-741d-4efb-bed4-88a738648c2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:07.000474 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d8104-741d-4efb-bed4-88a738648c2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:07.001889 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d8104-741d-4efb-bed4-88a738648c2c service cinder] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:07.012035 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1466/2926] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:05:06 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:05:07.151760 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8255842-571a-49d1-8afa-c9f55704365b None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:07.214676 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8255842-571a-49d1-8afa-c9f55704365b None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:07.219347 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8a372-ca28-4069-970f-68e2fc0b9019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:07.219970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8a372-ca28-4069-970f-68e2fc0b9019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:07.220247 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8a372-ca28-4069-970f-68e2fc0b9019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:07.220520 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8a372-ca28-4069-970f-68e2fc0b9019 admin admin] PATH_INFO: `/v3/users/7460021c09bc4d23970b5214d84f0c92` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:07.221639 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8a372-ca28-4069-970f-68e2fc0b9019 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7460021c09bc4d23970b5214d84f0c92)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:07.224206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8a372-ca28-4069-970f-68e2fc0b9019 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:07.406531 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-19f8a372-ca28-4069-970f-68e2fc0b9019 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7460021c09bc4d23970b5214d84f0c92'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:07.421214 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-19f8a372-ca28-4069-970f-68e2fc0b9019 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7460021c09bc4d23970b5214d84f0c92'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:07.428506 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-19f8a372-ca28-4069-970f-68e2fc0b9019 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7460021c09bc4d23970b5214d84f0c92'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:07.430159 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:07.482341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-19f8a372-ca28-4069-970f-68e2fc0b9019 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7460021c09bc4d23970b5214d84f0c92'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:07.482989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-19f8a372-ca28-4069-970f-68e2fc0b9019 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7460021c09bc4d23970b5214d84f0c92'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:07.489310 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1461/2927] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:05:07 2022] DELETE /identity/v3/users/7460021c09bc4d23970b5214d84f0c92 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:07.494484 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c212aa-5990-4e5a-b89d-0b648161e808 None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:07.494819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c212aa-5990-4e5a-b89d-0b648161e808 None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:07.495136 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c212aa-5990-4e5a-b89d-0b648161e808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:07.514055 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-84c212aa-5990-4e5a-b89d-0b648161e808 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:07.515576 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-84c212aa-5990-4e5a-b89d-0b648161e808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:07.574691 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1467/2928] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:07.585104 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8a372-ca28-4069-970f-68e2fc0b9019 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:07.586504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-19f8a372-ca28-4069-970f-68e2fc0b9019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:07.642764 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8a372-ca28-4069-970f-68e2fc0b9019 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:07.647045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c064fb4-6d70-45ca-827d-6d49e822e4fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:07.647683 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c064fb4-6d70-45ca-827d-6d49e822e4fe service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:07.647963 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c064fb4-6d70-45ca-827d-6d49e822e4fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:07.648233 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c064fb4-6d70-45ca-827d-6d49e822e4fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:07.668308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c064fb4-6d70-45ca-827d-6d49e822e4fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:07.668308 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c064fb4-6d70-45ca-827d-6d49e822e4fe service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:07.670286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1462/2929] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:05:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:05:08.175004 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c212aa-5990-4e5a-b89d-0b648161e808 None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:08.176605 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-84c212aa-5990-4e5a-b89d-0b648161e808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:08.190236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c212aa-5990-4e5a-b89d-0b648161e808 None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:08.195427 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00503376-d2f8-4ad7-87c1-f5e00a33a3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:08.197156 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00503376-d2f8-4ad7-87c1-f5e00a33a3db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:08.198799 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00503376-d2f8-4ad7-87c1-f5e00a33a3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:08.199049 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00503376-d2f8-4ad7-87c1-f5e00a33a3db admin admin] PATH_INFO: `/v3/projects/f229b5419d4c4a53a5e9689e96b0f87d` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:08.200357 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00503376-d2f8-4ad7-87c1-f5e00a33a3db admin admin] RBAC: Authorizing `identity:delete_project(project_id=f229b5419d4c4a53a5e9689e96b0f87d)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:08.202469 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00503376-d2f8-4ad7-87c1-f5e00a33a3db admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:08.317746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-00503376-d2f8-4ad7-87c1-f5e00a33a3db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f229b5419d4c4a53a5e9689e96b0f87d', 'request_id': 'req-00503376-d2f8-4ad7-87c1-f5e00a33a3db'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:08.432720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-00503376-d2f8-4ad7-87c1-f5e00a33a3db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f229b5419d4c4a53a5e9689e96b0f87d', 'request_id': 'req-00503376-d2f8-4ad7-87c1-f5e00a33a3db'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:08.440213 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-00503376-d2f8-4ad7-87c1-f5e00a33a3db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f229b5419d4c4a53a5e9689e96b0f87d', 'request_id': 'req-00503376-d2f8-4ad7-87c1-f5e00a33a3db'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:08.452057 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-00503376-d2f8-4ad7-87c1-f5e00a33a3db admin admin] The token cache is being invalidate because project f229b5419d4c4a53a5e9689e96b0f87d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:05:08.452242 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-00503376-d2f8-4ad7-87c1-f5e00a33a3db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:08.452928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-00503376-d2f8-4ad7-87c1-f5e00a33a3db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:08.454670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1468/2930] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:05:08 2022] DELETE /identity/v3/projects/f229b5419d4c4a53a5e9689e96b0f87d => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:08.459504 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c064fb4-6d70-45ca-827d-6d49e822e4fe service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:08.472039 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c064fb4-6d70-45ca-827d-6d49e822e4fe service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:08.557010 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8324b0-3b5e-41bc-8501-5b9aca5ce451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:08.557659 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8324b0-3b5e-41bc-8501-5b9aca5ce451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:08.557940 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8324b0-3b5e-41bc-8501-5b9aca5ce451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:08.558216 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8324b0-3b5e-41bc-8501-5b9aca5ce451 admin admin] PATH_INFO: `/v3/users/dcef8e47dda14870b99503e41aa54e2f` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:08.559397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8324b0-3b5e-41bc-8501-5b9aca5ce451 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcef8e47dda14870b99503e41aa54e2f)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:08.561318 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8324b0-3b5e-41bc-8501-5b9aca5ce451 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:08.783094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-0a8324b0-3b5e-41bc-8501-5b9aca5ce451 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcef8e47dda14870b99503e41aa54e2f'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:08.795220 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-0a8324b0-3b5e-41bc-8501-5b9aca5ce451 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcef8e47dda14870b99503e41aa54e2f'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:08.802546 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-0a8324b0-3b5e-41bc-8501-5b9aca5ce451 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcef8e47dda14870b99503e41aa54e2f'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:08.870020 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-0a8324b0-3b5e-41bc-8501-5b9aca5ce451 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcef8e47dda14870b99503e41aa54e2f'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:08.870939 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-0a8324b0-3b5e-41bc-8501-5b9aca5ce451 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcef8e47dda14870b99503e41aa54e2f'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:08.878320 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1463/2931] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:05:08 2022] DELETE /identity/v3/users/dcef8e47dda14870b99503e41aa54e2f => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:09.341423 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00503376-d2f8-4ad7-87c1-f5e00a33a3db admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:09.342913 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-00503376-d2f8-4ad7-87c1-f5e00a33a3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:09.359402 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00503376-d2f8-4ad7-87c1-f5e00a33a3db admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:09.401212 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:09.423525 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a69b77-87f3-4602-b6f2-d22b5d57c3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:09.424166 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a69b77-87f3-4602-b6f2-d22b5d57c3fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:09.424450 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a69b77-87f3-4602-b6f2-d22b5d57c3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:09.424728 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a69b77-87f3-4602-b6f2-d22b5d57c3fb admin admin] PATH_INFO: `/v3/projects/1fb11f72be4d4d74840075ee2cea0587` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:09.425867 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a69b77-87f3-4602-b6f2-d22b5d57c3fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fb11f72be4d4d74840075ee2cea0587)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:09.427824 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a69b77-87f3-4602-b6f2-d22b5d57c3fb admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:09.517060 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-51a69b77-87f3-4602-b6f2-d22b5d57c3fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fb11f72be4d4d74840075ee2cea0587', 'request_id': 'req-51a69b77-87f3-4602-b6f2-d22b5d57c3fb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:09.568128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-51a69b77-87f3-4602-b6f2-d22b5d57c3fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fb11f72be4d4d74840075ee2cea0587', 'request_id': 'req-51a69b77-87f3-4602-b6f2-d22b5d57c3fb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:09.574975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-51a69b77-87f3-4602-b6f2-d22b5d57c3fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fb11f72be4d4d74840075ee2cea0587', 'request_id': 'req-51a69b77-87f3-4602-b6f2-d22b5d57c3fb'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:09.585524 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-51a69b77-87f3-4602-b6f2-d22b5d57c3fb admin admin] The token cache is being invalidate because project 1fb11f72be4d4d74840075ee2cea0587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:05:09.585746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-51a69b77-87f3-4602-b6f2-d22b5d57c3fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:09.586387 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-51a69b77-87f3-4602-b6f2-d22b5d57c3fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:09.587904 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1469/2932] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:05:09 2022] DELETE /identity/v3/projects/1fb11f72be4d4d74840075ee2cea0587 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:09.592801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa90d92-d76e-4af7-b4f6-ec3e4347a0f0 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:09.593422 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa90d92-d76e-4af7-b4f6-ec3e4347a0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:09.593422 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa90d92-d76e-4af7-b4f6-ec3e4347a0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:09.613349 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-eaa90d92-d76e-4af7-b4f6-ec3e4347a0f0 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:09.614552 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-eaa90d92-d76e-4af7-b4f6-ec3e4347a0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:09.623687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:09.676180 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1464/2933] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:09.684332 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3243c-0fe2-4554-9f25-30b4cfb34bfc None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:09.684618 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3243c-0fe2-4554-9f25-30b4cfb34bfc None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:09.684899 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3243c-0fe2-4554-9f25-30b4cfb34bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:09.705435 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e1e3243c-0fe2-4554-9f25-30b4cfb34bfc None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:09.706692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e1e3243c-0fe2-4554-9f25-30b4cfb34bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:09.714950 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1470/2934] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:09.719674 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa90d92-d76e-4af7-b4f6-ec3e4347a0f0 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:09.729250 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa90d92-d76e-4af7-b4f6-ec3e4347a0f0 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:09.734344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8d27a-6774-4f0c-8014-145bf1d3d07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:09.734964 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8d27a-6774-4f0c-8014-145bf1d3d07c admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:09.735252 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8d27a-6774-4f0c-8014-145bf1d3d07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:09.735523 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8d27a-6774-4f0c-8014-145bf1d3d07c admin admin] PATH_INFO: `/v3/domains` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:09.736161 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8d27a-6774-4f0c-8014-145bf1d3d07c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:05:09.736462 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8d27a-6774-4f0c-8014-145bf1d3d07c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:09.738038 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8d27a-6774-4f0c-8014-145bf1d3d07c admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:09.750068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1465/2935] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:05:09.754473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3243c-0fe2-4554-9f25-30b4cfb34bfc None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:09.764370 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3243c-0fe2-4554-9f25-30b4cfb34bfc None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:09.769357 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94468f-6453-4d7f-93c3-23d5aac704bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:09.769976 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94468f-6453-4d7f-93c3-23d5aac704bc admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:09.770241 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94468f-6453-4d7f-93c3-23d5aac704bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:09.770516 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94468f-6453-4d7f-93c3-23d5aac704bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:09.771144 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94468f-6453-4d7f-93c3-23d5aac704bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:09.773060 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94468f-6453-4d7f-93c3-23d5aac704bc admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:09.838388 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1471/2936] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:05:09 2022] POST /identity/v3/projects => generated 405 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:05:09.842743 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8d27a-6774-4f0c-8014-145bf1d3d07c admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:09.853704 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8d27a-6774-4f0c-8014-145bf1d3d07c admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:09.910059 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94e1db-22e5-417b-91c0-b5650c941de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:09.910670 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94e1db-22e5-417b-91c0-b5650c941de4 admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:09.910953 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94e1db-22e5-417b-91c0-b5650c941de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:09.911261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94e1db-22e5-417b-91c0-b5650c941de4 admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:09.912591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94e1db-22e5-417b-91c0-b5650c941de4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:09.914568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94e1db-22e5-417b-91c0-b5650c941de4 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:09.975328 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1466/2937] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:05:09 2022] POST /identity/v3/users => generated 303 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:05:09.979810 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94468f-6453-4d7f-93c3-23d5aac704bc admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:09.990806 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94468f-6453-4d7f-93c3-23d5aac704bc admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:09.995444 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9dfcb-35b1-414d-ab81-daef1686271c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:09.996064 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9dfcb-35b1-414d-ab81-daef1686271c admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:09.996315 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9dfcb-35b1-414d-ab81-daef1686271c admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:09.996606 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9dfcb-35b1-414d-ab81-daef1686271c admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:09.997220 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede9dfcb-35b1-414d-ab81-daef1686271c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:09.998807 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede9dfcb-35b1-414d-ab81-daef1686271c admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:10.009265 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1472/2938] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:05:10.015284 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94e1db-22e5-417b-91c0-b5650c941de4 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:10.026389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94e1db-22e5-417b-91c0-b5650c941de4 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:10.031094 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02698d-4bce-4157-99ce-0d1896c246bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:10.031744 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02698d-4bce-4157-99ce-0d1896c246bf admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:10.032013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02698d-4bce-4157-99ce-0d1896c246bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:10.032286 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02698d-4bce-4157-99ce-0d1896c246bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:10.032935 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02698d-4bce-4157-99ce-0d1896c246bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:10.035107 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02698d-4bce-4157-99ce-0d1896c246bf admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:10.047482 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1467/2939] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:05:10.052097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9dfcb-35b1-414d-ab81-daef1686271c admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:10.062168 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9dfcb-35b1-414d-ab81-daef1686271c admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:10.066841 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcfe52-0216-46e7-9d65-c9b7ab36046d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:10.067523 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcfe52-0216-46e7-9d65-c9b7ab36046d admin admin] REQUEST_METHOD: `PUT` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:10.068454 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcfe52-0216-46e7-9d65-c9b7ab36046d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:10.068724 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcfe52-0216-46e7-9d65-c9b7ab36046d admin admin] PATH_INFO: `/v3/projects/38b0177c9907409da16648abdd2148cb/users/0e43cf23040d4e99aeae9ebd97803df4/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:10.084914 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babcfe52-0216-46e7-9d65-c9b7ab36046d admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b0177c9907409da16648abdd2148cb, user_id=0e43cf23040d4e99aeae9ebd97803df4, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:10.087604 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babcfe52-0216-46e7-9d65-c9b7ab36046d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:10.162767 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1473/2940] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:05:10 2022] PUT /identity/v3/projects/38b0177c9907409da16648abdd2148cb/users/0e43cf23040d4e99aeae9ebd97803df4/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:10.168044 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1180a69-92db-48f8-a437-d6f0b8ea9593 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:10.168456 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1180a69-92db-48f8-a437-d6f0b8ea9593 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:10.168797 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1180a69-92db-48f8-a437-d6f0b8ea9593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:10.195513 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-b1180a69-92db-48f8-a437-d6f0b8ea9593 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:10.196707 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b1180a69-92db-48f8-a437-d6f0b8ea9593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:10.256715 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1468/2941] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:10.266736 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcfe52-0216-46e7-9d65-c9b7ab36046d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:10.268437 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-babcfe52-0216-46e7-9d65-c9b7ab36046d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:10.325945 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcfe52-0216-46e7-9d65-c9b7ab36046d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:10.329606 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b8d46-1e20-4a71-a4b6-d1e223e8969f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:10.330220 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b8d46-1e20-4a71-a4b6-d1e223e8969f service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:10.330505 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b8d46-1e20-4a71-a4b6-d1e223e8969f service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:10.330781 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b8d46-1e20-4a71-a4b6-d1e223e8969f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:10.340836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b8d46-1e20-4a71-a4b6-d1e223e8969f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:10.342267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b8d46-1e20-4a71-a4b6-d1e223e8969f service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:10.352473 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1474/2942] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:05:11.674297 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c40ca-7091-4577-a6c8-dab6cba4cfc8 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:11.674802 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c40ca-7091-4577-a6c8-dab6cba4cfc8 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:11.674894 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c40ca-7091-4577-a6c8-dab6cba4cfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:11.705818 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-ac7c40ca-7091-4577-a6c8-dab6cba4cfc8 None None] MFA Rules not processed for user `0e43cf23040d4e99aeae9ebd97803df4`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:11.706966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-ac7c40ca-7091-4577-a6c8-dab6cba4cfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:11.716583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:11.777908 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1469/2943] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:05:11 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:11.782441 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b8d46-1e20-4a71-a4b6-d1e223e8969f service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:11.792720 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b8d46-1e20-4a71-a4b6-d1e223e8969f service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:11.797340 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc907c-6c29-4114-aef0-4c318b33b1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:11.797955 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc907c-6c29-4114-aef0-4c318b33b1c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:11.802879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc907c-6c29-4114-aef0-4c318b33b1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:11.802879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc907c-6c29-4114-aef0-4c318b33b1c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:11.802879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc907c-6c29-4114-aef0-4c318b33b1c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:11.802879 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc907c-6c29-4114-aef0-4c318b33b1c0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:11.807102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:11.898290 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1475/2944] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:05:11 2022] POST /identity/v3/projects => generated 405 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:05:11.900253 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c40ca-7091-4577-a6c8-dab6cba4cfc8 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:11.917712 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c40ca-7091-4577-a6c8-dab6cba4cfc8 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:11.991147 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb113d-c329-448a-af96-bb9b4fe503fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:11.991663 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb113d-c329-448a-af96-bb9b4fe503fb admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:11.991973 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb113d-c329-448a-af96-bb9b4fe503fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:11.992268 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb113d-c329-448a-af96-bb9b4fe503fb admin admin] PATH_INFO: `/v3/users` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:11.992906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb113d-c329-448a-af96-bb9b4fe503fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:11.994826 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb113d-c329-448a-af96-bb9b4fe503fb admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:12.101089 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1470/2945] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:05:11 2022] POST /identity/v3/users => generated 309 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 12:05:12.105236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc907c-6c29-4114-aef0-4c318b33b1c0 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:12.116433 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc907c-6c29-4114-aef0-4c318b33b1c0 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:12.121486 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d61cb-e9db-4438-b53d-97438a9a4ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:12.122110 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d61cb-e9db-4438-b53d-97438a9a4ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:12.122393 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d61cb-e9db-4438-b53d-97438a9a4ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:12.122670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d61cb-e9db-4438-b53d-97438a9a4ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:12.123462 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d61cb-e9db-4438-b53d-97438a9a4ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:12.125077 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d61cb-e9db-4438-b53d-97438a9a4ccf admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:12.135807 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1476/2946] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:05:12.140443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb113d-c329-448a-af96-bb9b4fe503fb admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:12.151610 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb113d-c329-448a-af96-bb9b4fe503fb admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:12.156165 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e047c-5ef8-4d6a-94a3-c12ba04ab406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:12.156817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e047c-5ef8-4d6a-94a3-c12ba04ab406 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:12.157091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e047c-5ef8-4d6a-94a3-c12ba04ab406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:12.157359 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e047c-5ef8-4d6a-94a3-c12ba04ab406 admin admin] PATH_INFO: `/v3/projects/6345d24be467455a9a917ed2743bee66/users/974f70978ae44c5da71237ea03f3a6e7/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:12.173133 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e047c-5ef8-4d6a-94a3-c12ba04ab406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6345d24be467455a9a917ed2743bee66, user_id=974f70978ae44c5da71237ea03f3a6e7, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:12.175841 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e047c-5ef8-4d6a-94a3-c12ba04ab406 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:12.340262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1471/2947] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:05:12 2022] PUT /identity/v3/projects/6345d24be467455a9a917ed2743bee66/users/974f70978ae44c5da71237ea03f3a6e7/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:12.344550 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d61cb-e9db-4438-b53d-97438a9a4ccf admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:12.354529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d61cb-e9db-4438-b53d-97438a9a4ccf admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:12.411346 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e7b92-41fc-4774-98be-7a4b6bc4e96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:12.411974 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e7b92-41fc-4774-98be-7a4b6bc4e96d admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:12.412248 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e7b92-41fc-4774-98be-7a4b6bc4e96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:12.412520 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e7b92-41fc-4774-98be-7a4b6bc4e96d admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:12.413203 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e7b92-41fc-4774-98be-7a4b6bc4e96d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:12.414804 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e7b92-41fc-4774-98be-7a4b6bc4e96d admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:12.425634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1477/2948] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 12:05:12.429947 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e047c-5ef8-4d6a-94a3-c12ba04ab406 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:12.441502 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e047c-5ef8-4d6a-94a3-c12ba04ab406 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:12.447091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110ebbc-d363-4bf7-9606-1be3095db108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:12.447817 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110ebbc-d363-4bf7-9606-1be3095db108 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:12.448099 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110ebbc-d363-4bf7-9606-1be3095db108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:12.448369 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110ebbc-d363-4bf7-9606-1be3095db108 admin admin] PATH_INFO: `/v3/projects/6345d24be467455a9a917ed2743bee66/users/974f70978ae44c5da71237ea03f3a6e7/roles/ee5bfb3e2bf64cf0847ac93b014672d1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:12.450487 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110ebbc-d363-4bf7-9606-1be3095db108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6345d24be467455a9a917ed2743bee66, user_id=974f70978ae44c5da71237ea03f3a6e7, role_id=ee5bfb3e2bf64cf0847ac93b014672d1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:12.453236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110ebbc-d363-4bf7-9606-1be3095db108 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:12.464833 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1472/2949] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:05:12 2022] PUT /identity/v3/projects/6345d24be467455a9a917ed2743bee66/users/974f70978ae44c5da71237ea03f3a6e7/roles/ee5bfb3e2bf64cf0847ac93b014672d1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:12.814740 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e7b92-41fc-4774-98be-7a4b6bc4e96d admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:12.876584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e7b92-41fc-4774-98be-7a4b6bc4e96d admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:12.926252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083b3c0-2e30-411c-a121-274c259638a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:12.926865 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083b3c0-2e30-411c-a121-274c259638a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:12.927148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083b3c0-2e30-411c-a121-274c259638a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:12.927458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083b3c0-2e30-411c-a121-274c259638a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:12.929770 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-f083b3c0-2e30-411c-a121-274c259638a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:12.980754 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:13.002873 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083b3c0-2e30-411c-a121-274c259638a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:13.004443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083b3c0-2e30-411c-a121-274c259638a3 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:13.015286 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1478/2950] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:05:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:05:14.099045 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ca8ef-f42f-499a-acd6-9617daa526d4 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:14.099633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ca8ef-f42f-499a-acd6-9617daa526d4 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:14.099633 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ca8ef-f42f-499a-acd6-9617daa526d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:14.135262 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-756ca8ef-f42f-499a-acd6-9617daa526d4 None None] MFA Rules not processed for user `974f70978ae44c5da71237ea03f3a6e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:14.136498 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-756ca8ef-f42f-499a-acd6-9617daa526d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:14.197061 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:14.218415 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1473/2951] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:05:14 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:15.896819 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083b3c0-2e30-411c-a121-274c259638a3 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:15.981750 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083b3c0-2e30-411c-a121-274c259638a3 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:15.986405 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0fabf-f816-4cdc-82da-89f90cc4d979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:15.987348 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0fabf-f816-4cdc-82da-89f90cc4d979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:15.987699 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0fabf-f816-4cdc-82da-89f90cc4d979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:15.988065 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0fabf-f816-4cdc-82da-89f90cc4d979 admin admin] PATH_INFO: `/v3/users/0e43cf23040d4e99aeae9ebd97803df4` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:15.989260 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0fabf-f816-4cdc-82da-89f90cc4d979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e43cf23040d4e99aeae9ebd97803df4)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:15.991374 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0fabf-f816-4cdc-82da-89f90cc4d979 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:16.205584 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4ef0fabf-f816-4cdc-82da-89f90cc4d979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e43cf23040d4e99aeae9ebd97803df4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:16.225544 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4ef0fabf-f816-4cdc-82da-89f90cc4d979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e43cf23040d4e99aeae9ebd97803df4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:16.239196 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4ef0fabf-f816-4cdc-82da-89f90cc4d979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e43cf23040d4e99aeae9ebd97803df4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:16.374208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4ef0fabf-f816-4cdc-82da-89f90cc4d979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e43cf23040d4e99aeae9ebd97803df4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:16.374833 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-4ef0fabf-f816-4cdc-82da-89f90cc4d979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e43cf23040d4e99aeae9ebd97803df4'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:16.382122 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1479/2952] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:05:15 2022] DELETE /identity/v3/users/0e43cf23040d4e99aeae9ebd97803df4 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:16.661537 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ca8ef-f42f-499a-acd6-9617daa526d4 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:16.682651 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ca8ef-f42f-499a-acd6-9617daa526d4 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:16.743970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69bb5a-8475-47c6-8c81-6efd77f5cd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:16.744619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69bb5a-8475-47c6-8c81-6efd77f5cd17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:16.744906 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69bb5a-8475-47c6-8c81-6efd77f5cd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:16.745186 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69bb5a-8475-47c6-8c81-6efd77f5cd17 admin admin] PATH_INFO: `/v3/projects/38b0177c9907409da16648abdd2148cb` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:16.746346 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db69bb5a-8475-47c6-8c81-6efd77f5cd17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b0177c9907409da16648abdd2148cb)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:16.748332 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db69bb5a-8475-47c6-8c81-6efd77f5cd17 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:16.882803 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-db69bb5a-8475-47c6-8c81-6efd77f5cd17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b0177c9907409da16648abdd2148cb', 'request_id': 'req-db69bb5a-8475-47c6-8c81-6efd77f5cd17'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:16.888172 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-db69bb5a-8475-47c6-8c81-6efd77f5cd17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b0177c9907409da16648abdd2148cb', 'request_id': 'req-db69bb5a-8475-47c6-8c81-6efd77f5cd17'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:16.967991 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-db69bb5a-8475-47c6-8c81-6efd77f5cd17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b0177c9907409da16648abdd2148cb', 'request_id': 'req-db69bb5a-8475-47c6-8c81-6efd77f5cd17'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:16.981249 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-db69bb5a-8475-47c6-8c81-6efd77f5cd17 admin admin] The token cache is being invalidate because project 38b0177c9907409da16648abdd2148cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:05:16.981542 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-db69bb5a-8475-47c6-8c81-6efd77f5cd17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:16.982194 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-db69bb5a-8475-47c6-8c81-6efd77f5cd17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:16.983858 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1474/2953] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:05:16 2022] DELETE /identity/v3/projects/38b0177c9907409da16648abdd2148cb => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:16.987918 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0fabf-f816-4cdc-82da-89f90cc4d979 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:16.989440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-4ef0fabf-f816-4cdc-82da-89f90cc4d979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:17.013396 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0fabf-f816-4cdc-82da-89f90cc4d979 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:17.055210 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:17.069993 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc9358-099d-4b36-ae78-af97c707fea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:17.071324 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc9358-099d-4b36-ae78-af97c707fea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:17.071324 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc9358-099d-4b36-ae78-af97c707fea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:17.071324 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc9358-099d-4b36-ae78-af97c707fea2 admin admin] PATH_INFO: `/v3/users/974f70978ae44c5da71237ea03f3a6e7` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:17.072373 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc9358-099d-4b36-ae78-af97c707fea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=974f70978ae44c5da71237ea03f3a6e7)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:17.074353 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc9358-099d-4b36-ae78-af97c707fea2 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:17.285060 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1edc9358-099d-4b36-ae78-af97c707fea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '974f70978ae44c5da71237ea03f3a6e7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:17.296897 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1edc9358-099d-4b36-ae78-af97c707fea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '974f70978ae44c5da71237ea03f3a6e7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:17.303975 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1edc9358-099d-4b36-ae78-af97c707fea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '974f70978ae44c5da71237ea03f3a6e7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:17.408532 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1edc9358-099d-4b36-ae78-af97c707fea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '974f70978ae44c5da71237ea03f3a6e7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:17.409161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-1edc9358-099d-4b36-ae78-af97c707fea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '974f70978ae44c5da71237ea03f3a6e7'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:17.417230 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1480/2954] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:05:16 2022] DELETE /identity/v3/users/974f70978ae44c5da71237ea03f3a6e7 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:17.666761 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69bb5a-8475-47c6-8c81-6efd77f5cd17 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:17.668296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-db69bb5a-8475-47c6-8c81-6efd77f5cd17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:17.682186 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69bb5a-8475-47c6-8c81-6efd77f5cd17 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:17.739901 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4adb6f-92a1-4648-9bd6-d90b725779d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:17.740535 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4adb6f-92a1-4648-9bd6-d90b725779d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:17.740812 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4adb6f-92a1-4648-9bd6-d90b725779d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:17.741083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4adb6f-92a1-4648-9bd6-d90b725779d3 admin admin] PATH_INFO: `/v3/projects/6345d24be467455a9a917ed2743bee66` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:17.742340 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4adb6f-92a1-4648-9bd6-d90b725779d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6345d24be467455a9a917ed2743bee66)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:17.744296 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4adb6f-92a1-4648-9bd6-d90b725779d3 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:17.761452 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:17.891049 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ad4adb6f-92a1-4648-9bd6-d90b725779d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6345d24be467455a9a917ed2743bee66', 'request_id': 'req-ad4adb6f-92a1-4648-9bd6-d90b725779d3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:17.896713 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ad4adb6f-92a1-4648-9bd6-d90b725779d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6345d24be467455a9a917ed2743bee66', 'request_id': 'req-ad4adb6f-92a1-4648-9bd6-d90b725779d3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:17.985195 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ad4adb6f-92a1-4648-9bd6-d90b725779d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6345d24be467455a9a917ed2743bee66', 'request_id': 'req-ad4adb6f-92a1-4648-9bd6-d90b725779d3'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:17.997351 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ad4adb6f-92a1-4648-9bd6-d90b725779d3 admin admin] The token cache is being invalidate because project 6345d24be467455a9a917ed2743bee66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:05:17.997637 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ad4adb6f-92a1-4648-9bd6-d90b725779d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:17.998259 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-ad4adb6f-92a1-4648-9bd6-d90b725779d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:17.999834 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1475/2955] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:05:17 2022] DELETE /identity/v3/projects/6345d24be467455a9a917ed2743bee66 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:18.005890 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46493ca4-e977-4296-bb6d-8c953911824b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:18.006386 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46493ca4-e977-4296-bb6d-8c953911824b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:18.006496 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46493ca4-e977-4296-bb6d-8c953911824b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:18.031458 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-46493ca4-e977-4296-bb6d-8c953911824b None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:18.032783 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-46493ca4-e977-4296-bb6d-8c953911824b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:18.093062 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1481/2956] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:18.101910 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a800a-4d65-4ae0-98a3-da737100b795 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:18.102197 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a800a-4d65-4ae0-98a3-da737100b795 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:18.102472 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a800a-4d65-4ae0-98a3-da737100b795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:18.124143 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-c82a800a-4d65-4ae0-98a3-da737100b795 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:18.125363 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-c82a800a-4d65-4ae0-98a3-da737100b795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:18.133838 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1476/2957] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:18.138148 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46493ca4-e977-4296-bb6d-8c953911824b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:18.148182 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46493ca4-e977-4296-bb6d-8c953911824b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:18.152847 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f991d-8f84-423f-872c-91654ec04a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:18.153448 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f991d-8f84-423f-872c-91654ec04a10 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:18.153757 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f991d-8f84-423f-872c-91654ec04a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:18.154034 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f991d-8f84-423f-872c-91654ec04a10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:18.154675 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f991d-8f84-423f-872c-91654ec04a10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:05:18.154983 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f991d-8f84-423f-872c-91654ec04a10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:18.156606 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f991d-8f84-423f-872c-91654ec04a10 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:18.168244 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1482/2958] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 12:05:18.172542 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a800a-4d65-4ae0-98a3-da737100b795 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:18.182389 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a800a-4d65-4ae0-98a3-da737100b795 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:18.187062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b999f-e28e-4281-ab65-c60bca0ea74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:18.187687 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b999f-e28e-4281-ab65-c60bca0ea74f admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:18.187966 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b999f-e28e-4281-ab65-c60bca0ea74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:18.188239 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b999f-e28e-4281-ab65-c60bca0ea74f admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:18.188900 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b999f-e28e-4281-ab65-c60bca0ea74f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:18.190779 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b999f-e28e-4281-ab65-c60bca0ea74f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:18.324580 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1477/2959] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:05:18 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:05:18.329394 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f991d-8f84-423f-872c-91654ec04a10 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:18.341692 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f991d-8f84-423f-872c-91654ec04a10 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:18.389146 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:18.399233 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cf7d5-2182-40fa-a59d-2e62b3a4cfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:18.399863 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cf7d5-2182-40fa-a59d-2e62b3a4cfb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:18.400126 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cf7d5-2182-40fa-a59d-2e62b3a4cfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:18.400403 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cf7d5-2182-40fa-a59d-2e62b3a4cfb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:18.401048 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cf7d5-2182-40fa-a59d-2e62b3a4cfb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:18.402964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cf7d5-2182-40fa-a59d-2e62b3a4cfb0 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:18.500456 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1483/2960] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:05:18 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:05:18.504931 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b999f-e28e-4281-ab65-c60bca0ea74f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:18.516244 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b999f-e28e-4281-ab65-c60bca0ea74f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:18.520934 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8677efb-dc71-4556-8790-e04cea833e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:18.521536 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8677efb-dc71-4556-8790-e04cea833e45 admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:18.521818 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8677efb-dc71-4556-8790-e04cea833e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:18.522093 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8677efb-dc71-4556-8790-e04cea833e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:18.522733 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8677efb-dc71-4556-8790-e04cea833e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:18.524365 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8677efb-dc71-4556-8790-e04cea833e45 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:18.534211 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1478/2961] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:05:18.538403 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cf7d5-2182-40fa-a59d-2e62b3a4cfb0 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:18.549143 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cf7d5-2182-40fa-a59d-2e62b3a4cfb0 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:18.554586 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7bf70-da1f-43e6-9f12-8e0d7e0468c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:18.555903 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7bf70-da1f-43e6-9f12-8e0d7e0468c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:18.556186 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7bf70-da1f-43e6-9f12-8e0d7e0468c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:18.556463 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7bf70-da1f-43e6-9f12-8e0d7e0468c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:18.557102 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca7bf70-da1f-43e6-9f12-8e0d7e0468c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:18.558690 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca7bf70-da1f-43e6-9f12-8e0d7e0468c1 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:18.569519 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1484/2962] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:05:18.573930 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8677efb-dc71-4556-8790-e04cea833e45 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:18.583987 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8677efb-dc71-4556-8790-e04cea833e45 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:18.589302 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bdaa8-db0c-4790-a117-564814010677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:18.589949 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bdaa8-db0c-4790-a117-564814010677 admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:18.590226 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bdaa8-db0c-4790-a117-564814010677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:18.590497 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bdaa8-db0c-4790-a117-564814010677 admin admin] PATH_INFO: `/v3/projects/f0bef0ea16d14445afb47e5917604ea3/users/8c6c4693b76f4f01bd975ccc83f87096/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:18.605564 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bdaa8-db0c-4790-a117-564814010677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0bef0ea16d14445afb47e5917604ea3, user_id=8c6c4693b76f4f01bd975ccc83f87096, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:18.608274 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bdaa8-db0c-4790-a117-564814010677 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:18.681784 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1479/2963] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:05:18 2022] PUT /identity/v3/projects/f0bef0ea16d14445afb47e5917604ea3/users/8c6c4693b76f4f01bd975ccc83f87096/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:18.746030 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2a311-4ca7-45d5-8d15-f083bb24159d None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:18.746321 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2a311-4ca7-45d5-8d15-f083bb24159d None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:18.746622 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2a311-4ca7-45d5-8d15-f083bb24159d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:18.774792 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-d4e2a311-4ca7-45d5-8d15-f083bb24159d None None] MFA Rules not processed for user `8c6c4693b76f4f01bd975ccc83f87096`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:18.776117 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-d4e2a311-4ca7-45d5-8d15-f083bb24159d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:18.839067 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1485/2964] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:05:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:18.849887 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bdaa8-db0c-4790-a117-564814010677 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:18.852236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-088bdaa8-db0c-4790-a117-564814010677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:18.915352 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bdaa8-db0c-4790-a117-564814010677 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:18.919009 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd89fb-f31b-44c6-b16c-461c9492b7d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558e3b6dd9754c57ad028aa431a68b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:18.919645 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd89fb-f31b-44c6-b16c-461c9492b7d7 service glance] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:18.919929 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd89fb-f31b-44c6-b16c-461c9492b7d7 service glance] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:18.920209 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd89fb-f31b-44c6-b16c-461c9492b7d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:18.928528 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd89fb-f31b-44c6-b16c-461c9492b7d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:18.930056 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd89fb-f31b-44c6-b16c-461c9492b7d7 service glance] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:18.938831 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1480/2965] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:05:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:05:28.835649 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=43,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:28.934791 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:37.317037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2a311-4ca7-45d5-8d15-f083bb24159d None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:37.378878 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2a311-4ca7-45d5-8d15-f083bb24159d None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:37.383446 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155914a4-1244-44c8-a122-b46150f1e5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:37.384076 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155914a4-1244-44c8-a122-b46150f1e5a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:37.384357 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155914a4-1244-44c8-a122-b46150f1e5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:37.384634 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155914a4-1244-44c8-a122-b46150f1e5a9 admin admin] PATH_INFO: `/v3/users/8c6c4693b76f4f01bd975ccc83f87096` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:37.385755 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155914a4-1244-44c8-a122-b46150f1e5a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c6c4693b76f4f01bd975ccc83f87096)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:37.387700 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155914a4-1244-44c8-a122-b46150f1e5a9 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:37.536318 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-155914a4-1244-44c8-a122-b46150f1e5a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c6c4693b76f4f01bd975ccc83f87096'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:37.548964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-155914a4-1244-44c8-a122-b46150f1e5a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c6c4693b76f4f01bd975ccc83f87096'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:37.556115 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-155914a4-1244-44c8-a122-b46150f1e5a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c6c4693b76f4f01bd975ccc83f87096'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:37.622779 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-155914a4-1244-44c8-a122-b46150f1e5a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c6c4693b76f4f01bd975ccc83f87096'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:37.623459 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-155914a4-1244-44c8-a122-b46150f1e5a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c6c4693b76f4f01bd975ccc83f87096'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:37.630331 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1486/2966] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:05:37 2022] DELETE /identity/v3/users/8c6c4693b76f4f01bd975ccc83f87096 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:37.634891 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c632a2-4479-4c8f-b2d9-93e733642585 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:37.635393 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c632a2-4479-4c8f-b2d9-93e733642585 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:37.635515 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c632a2-4479-4c8f-b2d9-93e733642585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:37.654230 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-42c632a2-4479-4c8f-b2d9-93e733642585 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:37.655521 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-42c632a2-4479-4c8f-b2d9-93e733642585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:37.712801 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1481/2967] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:37.722540 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155914a4-1244-44c8-a122-b46150f1e5a9 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:37.723962 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-155914a4-1244-44c8-a122-b46150f1e5a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:37.793179 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155914a4-1244-44c8-a122-b46150f1e5a9 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:37.796938 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55b567-7aa2-45ba-b99d-896e1aea2d3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:37.797535 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55b567-7aa2-45ba-b99d-896e1aea2d3c service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:37.797815 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55b567-7aa2-45ba-b99d-896e1aea2d3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:37.798083 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55b567-7aa2-45ba-b99d-896e1aea2d3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:37.807625 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55b567-7aa2-45ba-b99d-896e1aea2d3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:37.809097 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55b567-7aa2-45ba-b99d-896e1aea2d3c service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:37.818866 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1487/2968] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:05:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:05:38.213005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c632a2-4479-4c8f-b2d9-93e733642585 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:38.213005 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-42c632a2-4479-4c8f-b2d9-93e733642585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:38.226236 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c632a2-4479-4c8f-b2d9-93e733642585 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:38.231060 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99de79-8fb4-4464-bafe-bf9acb86bc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:38.231724 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99de79-8fb4-4464-bafe-bf9acb86bc9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:38.232006 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99de79-8fb4-4464-bafe-bf9acb86bc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:38.232309 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99de79-8fb4-4464-bafe-bf9acb86bc9a admin admin] PATH_INFO: `/v3/projects/f0bef0ea16d14445afb47e5917604ea3` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:38.233458 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99de79-8fb4-4464-bafe-bf9acb86bc9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0bef0ea16d14445afb47e5917604ea3)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:38.236186 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99de79-8fb4-4464-bafe-bf9acb86bc9a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:38.334162 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fa99de79-8fb4-4464-bafe-bf9acb86bc9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0bef0ea16d14445afb47e5917604ea3', 'request_id': 'req-fa99de79-8fb4-4464-bafe-bf9acb86bc9a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:38.340062 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fa99de79-8fb4-4464-bafe-bf9acb86bc9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0bef0ea16d14445afb47e5917604ea3', 'request_id': 'req-fa99de79-8fb4-4464-bafe-bf9acb86bc9a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:38.446430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fa99de79-8fb4-4464-bafe-bf9acb86bc9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0bef0ea16d14445afb47e5917604ea3', 'request_id': 'req-fa99de79-8fb4-4464-bafe-bf9acb86bc9a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:38.446430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fa99de79-8fb4-4464-bafe-bf9acb86bc9a admin admin] The token cache is being invalidate because project f0bef0ea16d14445afb47e5917604ea3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:05:38.446430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fa99de79-8fb4-4464-bafe-bf9acb86bc9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:38.446430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fa99de79-8fb4-4464-bafe-bf9acb86bc9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:05:38.446430 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1482/2969] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:05:38 2022] DELETE /identity/v3/projects/f0bef0ea16d14445afb47e5917604ea3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:05:38.446998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83866a-81ac-44c6-bbf5-00a96513252c None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:38.446998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83866a-81ac-44c6-bbf5-00a96513252c None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:38.446998 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83866a-81ac-44c6-bbf5-00a96513252c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:38.467670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-5b83866a-81ac-44c6-bbf5-00a96513252c None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:38.469429 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-5b83866a-81ac-44c6-bbf5-00a96513252c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:38.478434 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:38.529228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1488/2970] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:38.537618 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df0efe-fc82-4264-ad2a-934eea8e98c8 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:38.537904 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df0efe-fc82-4264-ad2a-934eea8e98c8 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:38.538180 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df0efe-fc82-4264-ad2a-934eea8e98c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:38.557758 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-59df0efe-fc82-4264-ad2a-934eea8e98c8 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:38.558970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-59df0efe-fc82-4264-ad2a-934eea8e98c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:38.566877 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1483/2971] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:38.571283 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83866a-81ac-44c6-bbf5-00a96513252c None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:38.581842 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83866a-81ac-44c6-bbf5-00a96513252c None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:38.587103 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52124d05-bd0f-4706-ba58-e2d51df0bc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:38.587732 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52124d05-bd0f-4706-ba58-e2d51df0bc54 admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:38.588036 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52124d05-bd0f-4706-ba58-e2d51df0bc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:38.588297 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52124d05-bd0f-4706-ba58-e2d51df0bc54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:38.588944 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52124d05-bd0f-4706-ba58-e2d51df0bc54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 12:05:38.589244 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52124d05-bd0f-4706-ba58-e2d51df0bc54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:38.590850 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52124d05-bd0f-4706-ba58-e2d51df0bc54 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:38.603029 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1489/2972] 10.222.0.78 () {58 vars in 1153 bytes} [Mon Sep 19 12:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 12:05:38.607396 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df0efe-fc82-4264-ad2a-934eea8e98c8 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:38.616778 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df0efe-fc82-4264-ad2a-934eea8e98c8 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:38.621725 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203ad2a-55fe-42d1-915e-e4581b35a4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:38.622356 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203ad2a-55fe-42d1-915e-e4581b35a4ad admin admin] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:38.622640 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203ad2a-55fe-42d1-915e-e4581b35a4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:38.622917 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203ad2a-55fe-42d1-915e-e4581b35a4ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:38.623600 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203ad2a-55fe-42d1-915e-e4581b35a4ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:38.626017 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203ad2a-55fe-42d1-915e-e4581b35a4ad admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:38.751568 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1484/2973] 10.222.0.78 () {60 vars in 1153 bytes} [Mon Sep 19 12:05:38 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:05:38.755849 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52124d05-bd0f-4706-ba58-e2d51df0bc54 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:38.766951 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52124d05-bd0f-4706-ba58-e2d51df0bc54 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:38.823101 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c777a1-815a-429c-85dd-cf5ed1dc81a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:38.823757 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c777a1-815a-429c-85dd-cf5ed1dc81a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:38.824041 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c777a1-815a-429c-85dd-cf5ed1dc81a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:38.824319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c777a1-815a-429c-85dd-cf5ed1dc81a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:38.824964 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c777a1-815a-429c-85dd-cf5ed1dc81a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:38.826880 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c777a1-815a-429c-85dd-cf5ed1dc81a3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:38.905748 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1490/2974] 10.222.0.78 () {60 vars in 1144 bytes} [Mon Sep 19 12:05:38 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 12:05:38.909806 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203ad2a-55fe-42d1-915e-e4581b35a4ad admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:38.920532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203ad2a-55fe-42d1-915e-e4581b35a4ad admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:38.925109 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f7213-e57b-4cf4-99fd-0010c712abeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:38.925730 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f7213-e57b-4cf4-99fd-0010c712abeb admin admin] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:38.926013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f7213-e57b-4cf4-99fd-0010c712abeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:38.926283 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f7213-e57b-4cf4-99fd-0010c712abeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:38.926916 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f7213-e57b-4cf4-99fd-0010c712abeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:38.928552 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f7213-e57b-4cf4-99fd-0010c712abeb admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:38.938965 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1485/2975] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:05:38.943017 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c777a1-815a-429c-85dd-cf5ed1dc81a3 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:38.953592 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c777a1-815a-429c-85dd-cf5ed1dc81a3 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:38.958315 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adefcb8-2a88-4b46-b48f-f8b42fd763ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:38.958914 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adefcb8-2a88-4b46-b48f-f8b42fd763ef admin admin] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:38.959226 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adefcb8-2a88-4b46-b48f-f8b42fd763ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:38.959505 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adefcb8-2a88-4b46-b48f-f8b42fd763ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:38.960142 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adefcb8-2a88-4b46-b48f-f8b42fd763ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:38.961717 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adefcb8-2a88-4b46-b48f-f8b42fd763ef admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:38.972208 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1491/2976] 10.222.0.78 () {58 vars in 1122 bytes} [Mon Sep 19 12:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 12:05:38.976372 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f7213-e57b-4cf4-99fd-0010c712abeb admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:38.986299 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f7213-e57b-4cf4-99fd-0010c712abeb admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:38.990770 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33248e-e4ac-444d-960b-1b729b68b36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:38.991443 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33248e-e4ac-444d-960b-1b729b68b36f admin admin] REQUEST_METHOD: `PUT` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:38.991720 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33248e-e4ac-444d-960b-1b729b68b36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:38.991989 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33248e-e4ac-444d-960b-1b729b68b36f admin admin] PATH_INFO: `/v3/projects/48567f32e12544ecbdc69ef98e2c7ff0/users/81472fea7c654abf99508f88d1e97839/roles/486614b0ab4b454b92e9ef0bc7ec61c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:39.006944 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33248e-e4ac-444d-960b-1b729b68b36f admin admin] RBAC: Authorizing `identity:create_grant(project_id=48567f32e12544ecbdc69ef98e2c7ff0, user_id=81472fea7c654abf99508f88d1e97839, role_id=486614b0ab4b454b92e9ef0bc7ec61c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:39.009583 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33248e-e4ac-444d-960b-1b729b68b36f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:39.116320 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1486/2977] 10.222.0.78 () {60 vars in 1483 bytes} [Mon Sep 19 12:05:38 2022] PUT /identity/v3/projects/48567f32e12544ecbdc69ef98e2c7ff0/users/81472fea7c654abf99508f88d1e97839/roles/486614b0ab4b454b92e9ef0bc7ec61c1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:05:39.128654 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8921c1-607d-4fa6-9106-7480399723aa None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:39.129037 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8921c1-607d-4fa6-9106-7480399723aa None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:39.129428 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8921c1-607d-4fa6-9106-7480399723aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:39.158324 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-de8921c1-607d-4fa6-9106-7480399723aa None None] MFA Rules not processed for user `81472fea7c654abf99508f88d1e97839`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:05:39.159606 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-de8921c1-607d-4fa6-9106-7480399723aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:39.227233 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1492/2978] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:05:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:05:39.239580 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33248e-e4ac-444d-960b-1b729b68b36f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:05:39.241289 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-cd33248e-e4ac-444d-960b-1b729b68b36f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:05:39.264920 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:39.302205 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33248e-e4ac-444d-960b-1b729b68b36f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:05:39.306069 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91765b21-9618-44af-9ab6-58cc6f450c1d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '558e3b6dd9754c57ad028aa431a68b80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:05:39.306678 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91765b21-9618-44af-9ab6-58cc6f450c1d service glance] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:05:39.306953 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91765b21-9618-44af-9ab6-58cc6f450c1d service glance] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:05:39.307271 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91765b21-9618-44af-9ab6-58cc6f450c1d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:05:39.316397 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91765b21-9618-44af-9ab6-58cc6f450c1d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:05:39.317830 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91765b21-9618-44af-9ab6-58cc6f450c1d service glance] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:05:39.326635 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1487/2979] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:05:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:05:49.223984 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:05:49.440316 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=17 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:06:07.548490 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8921c1-607d-4fa6-9106-7480399723aa None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:06:07.610375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8921c1-607d-4fa6-9106-7480399723aa None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:06:07.615461 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10953d89-1eaa-4bce-92d4-842696c9ebde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:06:07.616091 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10953d89-1eaa-4bce-92d4-842696c9ebde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:06:07.616375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10953d89-1eaa-4bce-92d4-842696c9ebde admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:06:07.616653 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10953d89-1eaa-4bce-92d4-842696c9ebde admin admin] PATH_INFO: `/v3/users/81472fea7c654abf99508f88d1e97839` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:06:07.617781 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10953d89-1eaa-4bce-92d4-842696c9ebde admin admin] RBAC: Authorizing `identity:delete_user(user_id=81472fea7c654abf99508f88d1e97839)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:06:07.619731 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10953d89-1eaa-4bce-92d4-842696c9ebde admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:06:07.790613 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-10953d89-1eaa-4bce-92d4-842696c9ebde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81472fea7c654abf99508f88d1e97839'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:06:07.803493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-10953d89-1eaa-4bce-92d4-842696c9ebde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81472fea7c654abf99508f88d1e97839'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:06:07.810470 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-10953d89-1eaa-4bce-92d4-842696c9ebde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81472fea7c654abf99508f88d1e97839'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:06:07.896304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-10953d89-1eaa-4bce-92d4-842696c9ebde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81472fea7c654abf99508f88d1e97839'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:06:07.896988 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-10953d89-1eaa-4bce-92d4-842696c9ebde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81472fea7c654abf99508f88d1e97839'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:06:07.903411 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1493/2980] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:06:07 2022] DELETE /identity/v3/users/81472fea7c654abf99508f88d1e97839 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:06:07.908113 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03706d71-111c-4fdc-9491-176e11393a20 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:06:07.908561 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03706d71-111c-4fdc-9491-176e11393a20 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:06:07.908561 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03706d71-111c-4fdc-9491-176e11393a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:06:07.928542 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-03706d71-111c-4fdc-9491-176e11393a20 None None] MFA Rules not processed for user `b83cd9cf7ce84815942f0d89b245aefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:06:07.929755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-03706d71-111c-4fdc-9491-176e11393a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:06:07.987619 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1488/2981] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:06:07.998161 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10953d89-1eaa-4bce-92d4-842696c9ebde admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:06:07.999616 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-10953d89-1eaa-4bce-92d4-842696c9ebde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:06:08.064038 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10953d89-1eaa-4bce-92d4-842696c9ebde admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:06:08.067654 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d8982-dab5-452f-b19d-c8e07f373510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:06:08.068271 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d8982-dab5-452f-b19d-c8e07f373510 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:06:08.068551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d8982-dab5-452f-b19d-c8e07f373510 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:06:08.068825 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d8982-dab5-452f-b19d-c8e07f373510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:06:08.078051 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d8982-dab5-452f-b19d-c8e07f373510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:06:08.079493 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d8982-dab5-452f-b19d-c8e07f373510 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:06:08.089495 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1494/2982] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:06:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:06:08.506453 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03706d71-111c-4fdc-9491-176e11393a20 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:06:08.507933 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-03706d71-111c-4fdc-9491-176e11393a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:06:08.521048 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03706d71-111c-4fdc-9491-176e11393a20 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:06:08.525926 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2fa21-532c-44e5-964c-6d7da67a3c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:06:08.526576 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2fa21-532c-44e5-964c-6d7da67a3c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:06:08.526879 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2fa21-532c-44e5-964c-6d7da67a3c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:06:08.527153 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2fa21-532c-44e5-964c-6d7da67a3c1a admin admin] PATH_INFO: `/v3/projects/48567f32e12544ecbdc69ef98e2c7ff0` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:06:08.528344 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2fa21-532c-44e5-964c-6d7da67a3c1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=48567f32e12544ecbdc69ef98e2c7ff0)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:06:08.530261 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2fa21-532c-44e5-964c-6d7da67a3c1a admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:06:08.630709 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fab2fa21-532c-44e5-964c-6d7da67a3c1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48567f32e12544ecbdc69ef98e2c7ff0', 'request_id': 'req-fab2fa21-532c-44e5-964c-6d7da67a3c1a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:06:08.636537 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fab2fa21-532c-44e5-964c-6d7da67a3c1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48567f32e12544ecbdc69ef98e2c7ff0', 'request_id': 'req-fab2fa21-532c-44e5-964c-6d7da67a3c1a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:06:08.736437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fab2fa21-532c-44e5-964c-6d7da67a3c1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48567f32e12544ecbdc69ef98e2c7ff0', 'request_id': 'req-fab2fa21-532c-44e5-964c-6d7da67a3c1a'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:06:08.748567 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fab2fa21-532c-44e5-964c-6d7da67a3c1a admin admin] The token cache is being invalidate because project 48567f32e12544ecbdc69ef98e2c7ff0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81738) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:06:08.748852 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fab2fa21-532c-44e5-964c-6d7da67a3c1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:06:08.749499 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-fab2fa21-532c-44e5-964c-6d7da67a3c1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:06:08.751091 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1489/2983] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:06:08 2022] DELETE /identity/v3/projects/48567f32e12544ecbdc69ef98e2c7ff0 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:06:18.084844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:06:18.745444 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:06:41.724400 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d8982-dab5-452f-b19d-c8e07f373510 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:06:41.725943 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-750d8982-dab5-452f-b19d-c8e07f373510 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:06:41.737837 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d8982-dab5-452f-b19d-c8e07f373510 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:06:41.789357 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e3bf0-e595-42f5-80ee-e36142071a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:06:41.789956 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e3bf0-e595-42f5-80ee-e36142071a79 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:06:41.790246 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e3bf0-e595-42f5-80ee-e36142071a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:06:41.790518 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e3bf0-e595-42f5-80ee-e36142071a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:06:41.792497 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e99e3bf0-e595-42f5-80ee-e36142071a79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:06:41.802212 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e3bf0-e595-42f5-80ee-e36142071a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:06:41.803670 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e3bf0-e595-42f5-80ee-e36142071a79 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:06:41.811808 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1495/2984] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:06:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:06:51.808152 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=26 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:07:13.881206 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2fa21-532c-44e5-964c-6d7da67a3c1a admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:07:13.882694 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-fab2fa21-532c-44e5-964c-6d7da67a3c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:07:13.895535 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2fa21-532c-44e5-964c-6d7da67a3c1a admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:07:13.899157 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc6277-117f-4f76-8fa8-71344b6fe3dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:07:13.899780 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc6277-117f-4f76-8fa8-71344b6fe3dc service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:07:13.900055 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc6277-117f-4f76-8fa8-71344b6fe3dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:07:13.900320 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc6277-117f-4f76-8fa8-71344b6fe3dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:07:13.902003 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-dcdc6277-117f-4f76-8fa8-71344b6fe3dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:07:13.979979 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc6277-117f-4f76-8fa8-71344b6fe3dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:07:13.981409 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc6277-117f-4f76-8fa8-71344b6fe3dc service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:07:14.002711 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1490/2985] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:07:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:07:15.193236 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e3bf0-e595-42f5-80ee-e36142071a79 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:07:15.201935 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e3bf0-e595-42f5-80ee-e36142071a79 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:07:15.205708 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b417426-7afa-43f7-9ba0-14b4b077ad37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:07:15.206347 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b417426-7afa-43f7-9ba0-14b4b077ad37 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:07:15.206632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b417426-7afa-43f7-9ba0-14b4b077ad37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:07:15.206911 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b417426-7afa-43f7-9ba0-14b4b077ad37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:07:15.208725 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-8b417426-7afa-43f7-9ba0-14b4b077ad37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:07:15.223066 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b417426-7afa-43f7-9ba0-14b4b077ad37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:07:15.224497 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b417426-7afa-43f7-9ba0-14b4b077ad37 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:07:15.236188 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1496/2986] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:07:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:07:16.297751 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc6277-117f-4f76-8fa8-71344b6fe3dc service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:07:16.301595 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-dcdc6277-117f-4f76-8fa8-71344b6fe3dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:07:16.316668 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc6277-117f-4f76-8fa8-71344b6fe3dc service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:07:16.322649 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cda0ee-34d8-4f39-9d41-73f2f9eb94c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:07:16.323309 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cda0ee-34d8-4f39-9d41-73f2f9eb94c8 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:07:16.323590 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cda0ee-34d8-4f39-9d41-73f2f9eb94c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:07:16.323865 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cda0ee-34d8-4f39-9d41-73f2f9eb94c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:07:16.325691 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-90cda0ee-34d8-4f39-9d41-73f2f9eb94c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:07:16.338755 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cda0ee-34d8-4f39-9d41-73f2f9eb94c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:07:16.344963 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cda0ee-34d8-4f39-9d41-73f2f9eb94c8 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:07:16.352706 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1491/2987] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:07:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:07:18.934722 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959822d-8969-4ca9-a20e-a5b44f29d94b None None] REQUEST_METHOD: `POST` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:07:18.935262 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959822d-8969-4ca9-a20e-a5b44f29d94b None None] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:07:18.935375 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959822d-8969-4ca9-a20e-a5b44f29d94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:07:18.954494 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.auth.core [None req-e959822d-8969-4ca9-a20e-a5b44f29d94b None None] MFA Rules not processed for user `ab1e5eb74e784f53a9250b9a7a2220c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:07:18.955742 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-e959822d-8969-4ca9-a20e-a5b44f29d94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:07:19.010050 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1497/2988] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:07:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:07:20.031234 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cda0ee-34d8-4f39-9d41-73f2f9eb94c8 service nova] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:07:20.032786 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-90cda0ee-34d8-4f39-9d41-73f2f9eb94c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:07:20.103970 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cda0ee-34d8-4f39-9d41-73f2f9eb94c8 service nova] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:07:20.109035 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be4977-2e7c-4ad1-8932-23e435caf04d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:07:20.109647 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be4977-2e7c-4ad1-8932-23e435caf04d service placement] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:07:20.109927 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be4977-2e7c-4ad1-8932-23e435caf04d service placement] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:07:20.110204 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be4977-2e7c-4ad1-8932-23e435caf04d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:07:20.112553 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-b6be4977-2e7c-4ad1-8932-23e435caf04d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:07:20.124645 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be4977-2e7c-4ad1-8932-23e435caf04d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:07:20.126258 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be4977-2e7c-4ad1-8932-23e435caf04d service placement] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:07:20.137068 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1492/2989] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:07:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:07:29.003060 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=31 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:07:30.132408 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=79 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:07:33.267997 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959822d-8969-4ca9-a20e-a5b44f29d94b None None] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:07:33.281031 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959822d-8969-4ca9-a20e-a5b44f29d94b None None] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:07:33.288712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10a520-91f8-4efd-ba69-53f593dfc8ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:07:33.288712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10a520-91f8-4efd-ba69-53f593dfc8ef service nova] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:07:33.288712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10a520-91f8-4efd-ba69-53f593dfc8ef service nova] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:07:33.288712 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10a520-91f8-4efd-ba69-53f593dfc8ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:07:33.289431 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9a10a520-91f8-4efd-ba69-53f593dfc8ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:07:33.302319 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10a520-91f8-4efd-ba69-53f593dfc8ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:07:33.303928 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10a520-91f8-4efd-ba69-53f593dfc8ef service nova] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:07:33.314551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1498/2990] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:07:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:07:37.350895 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be4977-2e7c-4ad1-8932-23e435caf04d service placement] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:07:37.364156 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be4977-2e7c-4ad1-8932-23e435caf04d service placement] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:07:37.369728 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2d7cc-6243-4918-ac08-768074d9ea49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aba1968529574fe5b4c5d114c874f428', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:07:37.370273 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2d7cc-6243-4918-ac08-768074d9ea49 service nova] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:07:37.370548 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2d7cc-6243-4918-ac08-768074d9ea49 service nova] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:07:37.370822 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2d7cc-6243-4918-ac08-768074d9ea49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:07:37.372594 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-dcc2d7cc-6243-4918-ac08-768074d9ea49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:07:37.386503 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2d7cc-6243-4918-ac08-768074d9ea49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:07:37.388639 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2d7cc-6243-4918-ac08-768074d9ea49 service nova] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:07:37.400392 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1493/2991] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:07:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:07:43.310055 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=6 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:07:47.396057 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=6 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:08:27.331046 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10a520-91f8-4efd-ba69-53f593dfc8ef service nova] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:08:27.332482 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9a10a520-91f8-4efd-ba69-53f593dfc8ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:08:27.346844 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10a520-91f8-4efd-ba69-53f593dfc8ef service nova] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:08:27.351970 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc3d2d-7512-4414-9142-0da7c2fa8527 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:08:27.353130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc3d2d-7512-4414-9142-0da7c2fa8527 service placement] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:08:27.353130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc3d2d-7512-4414-9142-0da7c2fa8527 service placement] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:08:27.353130 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc3d2d-7512-4414-9142-0da7c2fa8527 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:08:27.355460 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-2dfc3d2d-7512-4414-9142-0da7c2fa8527 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:08:27.367109 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc3d2d-7512-4414-9142-0da7c2fa8527 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:08:27.368701 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc3d2d-7512-4414-9142-0da7c2fa8527 service placement] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:08:27.379529 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1499/2992] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:08:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:08:37.375239 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=6 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:09:13.021338 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417d7b6-37e3-4308-9b6f-58bcfe666721 None None] REQUEST_METHOD: `POST` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:09:13.021889 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417d7b6-37e3-4308-9b6f-58bcfe666721 None None] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:09:13.021889 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417d7b6-37e3-4308-9b6f-58bcfe666721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:09:13.041363 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.auth.core [None req-e417d7b6-37e3-4308-9b6f-58bcfe666721 None None] MFA Rules not processed for user `ab1e5eb74e784f53a9250b9a7a2220c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 12:09:13.042558 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e417d7b6-37e3-4308-9b6f-58bcfe666721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:09:13.049558 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1494/2993] 10.222.0.78 () {58 vars in 958 bytes} [Mon Sep 19 12:09:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 12:09:13.383816 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc3d2d-7512-4414-9142-0da7c2fa8527 service placement] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:09:13.393828 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc3d2d-7512-4414-9142-0da7c2fa8527 service placement] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:09:13.397389 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6636d-21d8-4101-9605-3b8732a134b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:09:13.397991 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6636d-21d8-4101-9605-3b8732a134b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:09:13.398273 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6636d-21d8-4101-9605-3b8732a134b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:09:13.398551 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6636d-21d8-4101-9605-3b8732a134b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:09:13.400366 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-dce6636d-21d8-4101-9605-3b8732a134b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:09:13.410471 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6636d-21d8-4101-9605-3b8732a134b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:09:13.413252 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6636d-21d8-4101-9605-3b8732a134b0 service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:09:13.420323 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1500/2994] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:09:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:09:23.036515 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=3 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:09:23.416521 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=6 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:11:01.245591 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417d7b6-37e3-4308-9b6f-58bcfe666721 None None] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:11:01.256034 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417d7b6-37e3-4308-9b6f-58bcfe666721 None None] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:11:01.259659 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37ee5b-736b-456c-8435-22b4c8bedc17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:11:01.260264 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37ee5b-736b-456c-8435-22b4c8bedc17 service neutron] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:11:01.260544 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37ee5b-736b-456c-8435-22b4c8bedc17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:11:01.260825 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37ee5b-736b-456c-8435-22b4c8bedc17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:11:01.262555 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-dd37ee5b-736b-456c-8435-22b4c8bedc17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:11:01.273573 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37ee5b-736b-456c-8435-22b4c8bedc17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:11:01.275004 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37ee5b-736b-456c-8435-22b4c8bedc17 service neutron] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:11:01.284828 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1495/2995] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:11:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:11:11.280006 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=6 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:11:15.592128 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6636d-21d8-4101-9605-3b8732a134b0 service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:11:15.601622 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6636d-21d8-4101-9605-3b8732a134b0 service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:11:15.605321 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c540521-7810-4bc0-beb6-3afcf72fe6ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf0049a3e3a442e945f6207477a7fd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:11:15.605920 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c540521-7810-4bc0-beb6-3afcf72fe6ad service neutron] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:11:15.606199 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c540521-7810-4bc0-beb6-3afcf72fe6ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:11:15.606476 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c540521-7810-4bc0-beb6-3afcf72fe6ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:11:15.608251 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9c540521-7810-4bc0-beb6-3afcf72fe6ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:11:15.696902 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c540521-7810-4bc0-beb6-3afcf72fe6ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:11:15.698323 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c540521-7810-4bc0-beb6-3afcf72fe6ad service neutron] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:11:15.717836 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1501/2996] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:11:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 12:11:19.211266 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37ee5b-736b-456c-8435-22b4c8bedc17 service neutron] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:11:19.223436 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37ee5b-736b-456c-8435-22b4c8bedc17 service neutron] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:11:19.228543 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61039a0-5870-4451-a742-7eccef485681 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:11:19.229112 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61039a0-5870-4451-a742-7eccef485681 service placement] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:11:19.229390 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61039a0-5870-4451-a742-7eccef485681 service placement] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:11:19.229661 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61039a0-5870-4451-a742-7eccef485681 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:11:19.232077 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d61039a0-5870-4451-a742-7eccef485681 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:11:19.244013 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61039a0-5870-4451-a742-7eccef485681 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:11:19.251423 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61039a0-5870-4451-a742-7eccef485681 service placement] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:11:19.258161 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1496/2997] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:11:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:11:22.863267 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c540521-7810-4bc0-beb6-3afcf72fe6ad service neutron] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:11:22.885746 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c540521-7810-4bc0-beb6-3afcf72fe6ad service neutron] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:11:22.893433 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80ffc0-a26d-4699-bc46-a81eb9c490eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:11:22.895606 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80ffc0-a26d-4699-bc46-a81eb9c490eb service placement] REQUEST_METHOD: `GET` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:11:22.895906 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80ffc0-a26d-4699-bc46-a81eb9c490eb service placement] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:11:22.896184 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80ffc0-a26d-4699-bc46-a81eb9c490eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:11:22.899958 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-9f80ffc0-a26d-4699-bc46-a81eb9c490eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:11:22.913701 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f80ffc0-a26d-4699-bc46-a81eb9c490eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:11:22.915480 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f80ffc0-a26d-4699-bc46-a81eb9c490eb service placement] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:11:22.931443 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1502/2998] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:11:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:11:23.015304 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61039a0-5870-4451-a742-7eccef485681 service placement] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:11:23.016277 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-d61039a0-5870-4451-a742-7eccef485681 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:11:23.032532 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61039a0-5870-4451-a742-7eccef485681 service placement] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:11:23.037097 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a528a0-d695-4e18-bcbb-96ab0d3a550f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:11:23.037750 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a528a0-d695-4e18-bcbb-96ab0d3a550f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:11:23.038027 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a528a0-d695-4e18-bcbb-96ab0d3a550f admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:11:23.042294 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a528a0-d695-4e18-bcbb-96ab0d3a550f admin admin] PATH_INFO: `/v3/users/ab1e5eb74e784f53a9250b9a7a2220c1` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:11:23.042294 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a528a0-d695-4e18-bcbb-96ab0d3a550f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab1e5eb74e784f53a9250b9a7a2220c1)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:11:23.042294 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a528a0-d695-4e18-bcbb-96ab0d3a550f admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:11:23.256447 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e6a528a0-d695-4e18-bcbb-96ab0d3a550f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab1e5eb74e784f53a9250b9a7a2220c1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:23.269185 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e6a528a0-d695-4e18-bcbb-96ab0d3a550f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab1e5eb74e784f53a9250b9a7a2220c1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:23.276390 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e6a528a0-d695-4e18-bcbb-96ab0d3a550f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab1e5eb74e784f53a9250b9a7a2220c1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:23.341814 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e6a528a0-d695-4e18-bcbb-96ab0d3a550f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab1e5eb74e784f53a9250b9a7a2220c1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:23.342425 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-e6a528a0-d695-4e18-bcbb-96ab0d3a550f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab1e5eb74e784f53a9250b9a7a2220c1'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:23.349680 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1497/2999] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:11:23 2022] DELETE /identity/v3/users/ab1e5eb74e784f53a9250b9a7a2220c1 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:11:23.601304 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80ffc0-a26d-4699-bc46-a81eb9c490eb service placement] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:11:23.614912 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80ffc0-a26d-4699-bc46-a81eb9c490eb service placement] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:11:23.676800 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f14280-accc-4a46-9792-4ed17b2e35f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:11:23.677437 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f14280-accc-4a46-9792-4ed17b2e35f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:11:23.677722 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f14280-accc-4a46-9792-4ed17b2e35f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:11:23.678003 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f14280-accc-4a46-9792-4ed17b2e35f3 admin admin] PATH_INFO: `/v3/projects/b7d811c8ed584ddb92b4642d85e8db00` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:11:23.679132 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f14280-accc-4a46-9792-4ed17b2e35f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7d811c8ed584ddb92b4642d85e8db00)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:11:23.681105 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f14280-accc-4a46-9792-4ed17b2e35f3 admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:11:23.870875 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-13f14280-accc-4a46-9792-4ed17b2e35f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7d811c8ed584ddb92b4642d85e8db00', 'request_id': 'req-13f14280-accc-4a46-9792-4ed17b2e35f3'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:23.874688 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:11:23.995440 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-13f14280-accc-4a46-9792-4ed17b2e35f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7d811c8ed584ddb92b4642d85e8db00', 'request_id': 'req-13f14280-accc-4a46-9792-4ed17b2e35f3'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:24.003632 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-13f14280-accc-4a46-9792-4ed17b2e35f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7d811c8ed584ddb92b4642d85e8db00', 'request_id': 'req-13f14280-accc-4a46-9792-4ed17b2e35f3'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:24.014209 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-13f14280-accc-4a46-9792-4ed17b2e35f3 admin admin] The token cache is being invalidate because project b7d811c8ed584ddb92b4642d85e8db00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:11:24.014504 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-13f14280-accc-4a46-9792-4ed17b2e35f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:24.015154 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-13f14280-accc-4a46-9792-4ed17b2e35f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:24.017345 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1503/3000] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:11:23 2022] DELETE /identity/v3/projects/b7d811c8ed584ddb92b4642d85e8db00 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 12:11:24.021607 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a528a0-d695-4e18-bcbb-96ab0d3a550f admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:11:24.023066 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-e6a528a0-d695-4e18-bcbb-96ab0d3a550f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:11:24.036016 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a528a0-d695-4e18-bcbb-96ab0d3a550f admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:11:24.093741 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285138a9-af5a-4576-a8f9-0c2528f8b1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:11:24.094401 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285138a9-af5a-4576-a8f9-0c2528f8b1c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:11:24.094665 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285138a9-af5a-4576-a8f9-0c2528f8b1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:11:24.094946 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285138a9-af5a-4576-a8f9-0c2528f8b1c2 admin admin] PATH_INFO: `/v3/users/44eab873d87949369e77d39ff32507fa` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:11:24.096131 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285138a9-af5a-4576-a8f9-0c2528f8b1c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44eab873d87949369e77d39ff32507fa)` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:11:24.098195 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285138a9-af5a-4576-a8f9-0c2528f8b1c2 admin admin] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:11:24.267408 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-285138a9-af5a-4576-a8f9-0c2528f8b1c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44eab873d87949369e77d39ff32507fa'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:24.278089 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:11:24.280437 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-285138a9-af5a-4576-a8f9-0c2528f8b1c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44eab873d87949369e77d39ff32507fa'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:24.288845 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-285138a9-af5a-4576-a8f9-0c2528f8b1c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44eab873d87949369e77d39ff32507fa'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:24.361083 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-285138a9-af5a-4576-a8f9-0c2528f8b1c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44eab873d87949369e77d39ff32507fa'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:24.361712 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.notifications [None req-285138a9-af5a-4576-a8f9-0c2528f8b1c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44eab873d87949369e77d39ff32507fa'} {{(pid=81738) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:24.367976 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1498/3001] 10.222.0.78 () {58 vars in 1224 bytes} [Mon Sep 19 12:11:24 2022] DELETE /identity/v3/users/44eab873d87949369e77d39ff32507fa => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:11:24.597254 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f14280-accc-4a46-9792-4ed17b2e35f3 admin admin] Authenticating user token {{(pid=81737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:11:24.598727 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.fernet_utils [None req-13f14280-accc-4a46-9792-4ed17b2e35f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:11:24.614965 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f14280-accc-4a46-9792-4ed17b2e35f3 admin admin] Validating token access rules against request {{(pid=81737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:11:24.681747 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfacaed8-a9cb-4ab1-a2e0-477e9f35744f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83cd9cf7ce84815942f0d89b245aefe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a875b13c6f2b4a43a97ce59e85aa99fd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:11:24.682713 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacaed8-a9cb-4ab1-a2e0-477e9f35744f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:11:24.682997 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacaed8-a9cb-4ab1-a2e0-477e9f35744f admin admin] SCRIPT_NAME: `/identity` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:11:24.683301 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacaed8-a9cb-4ab1-a2e0-477e9f35744f admin admin] PATH_INFO: `/v3/projects/f7bb01d8ee844b70969677665fb154c8` {{(pid=81737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:11:24.684400 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfacaed8-a9cb-4ab1-a2e0-477e9f35744f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7bb01d8ee844b70969677665fb154c8)` {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:11:24.686297 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfacaed8-a9cb-4ab1-a2e0-477e9f35744f admin admin] RBAC: Authorization granted {{(pid=81737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:11:24.794753 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-dfacaed8-a9cb-4ab1-a2e0-477e9f35744f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7bb01d8ee844b70969677665fb154c8', 'request_id': 'req-dfacaed8-a9cb-4ab1-a2e0-477e9f35744f'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:24.879194 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-dfacaed8-a9cb-4ab1-a2e0-477e9f35744f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7bb01d8ee844b70969677665fb154c8', 'request_id': 'req-dfacaed8-a9cb-4ab1-a2e0-477e9f35744f'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:24.886062 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-dfacaed8-a9cb-4ab1-a2e0-477e9f35744f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7bb01d8ee844b70969677665fb154c8', 'request_id': 'req-dfacaed8-a9cb-4ab1-a2e0-477e9f35744f'} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:24.897228 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-dfacaed8-a9cb-4ab1-a2e0-477e9f35744f admin admin] The token cache is being invalidate because project f7bb01d8ee844b70969677665fb154c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81737) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 12:11:24.900578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-dfacaed8-a9cb-4ab1-a2e0-477e9f35744f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:24.900578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG keystone.notifications [None req-dfacaed8-a9cb-4ab1-a2e0-477e9f35744f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81737) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 12:11:24.900578 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: [pid: 81737|app: 0|req: 1504/3002] 10.222.0.78 () {58 vars in 1233 bytes} [Mon Sep 19 12:11:24 2022] DELETE /identity/v3/projects/f7bb01d8ee844b70969677665fb154c8 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 12:11:34.440689 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:11:34.894060 dv-cinder-822822-7-iscsi devstack@keystone.service[81737]: DEBUG dbcounter [-] [81737] Writing DB stats keystone:SELECT=57,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 12:13:31.166190 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285138a9-af5a-4576-a8f9-0c2528f8b1c2 admin admin] Authenticating user token {{(pid=81738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 12:13:31.167660 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-285138a9-af5a-4576-a8f9-0c2528f8b1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:13:31.290571 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285138a9-af5a-4576-a8f9-0c2528f8b1c2 admin admin] Validating token access rules against request {{(pid=81738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 12:13:31.349120 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e00ef1-f79d-45d2-8482-fc173566958b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0427cdd3905a422cab4fed0aac187ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8563e86b7c94ff29e39fb20a7969551', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 12:13:31.349646 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e00ef1-f79d-45d2-8482-fc173566958b service placement] REQUEST_METHOD: `GET` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 12:13:31.349925 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e00ef1-f79d-45d2-8482-fc173566958b service placement] SCRIPT_NAME: `/identity` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 12:13:31.350198 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e00ef1-f79d-45d2-8482-fc173566958b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 12:13:31.351972 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.fernet_utils [None req-96e00ef1-f79d-45d2-8482-fc173566958b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 12:13:31.363807 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e00ef1-f79d-45d2-8482-fc173566958b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 12:13:31.365431 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e00ef1-f79d-45d2-8482-fc173566958b service placement] RBAC: Authorization granted {{(pid=81738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 12:13:31.375816 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: [pid: 81738|app: 0|req: 1499/3003] 10.222.0.78 () {60 vars in 1365 bytes} [Mon Sep 19 12:13:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 12:13:41.371341 dv-cinder-822822-7-iscsi devstack@keystone.service[81738]: DEBUG dbcounter [-] [81738] Writing DB stats keystone:SELECT=80 {{(pid=81738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}